2024-04-26T18:05:42.161 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T18:05:42.161 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --verbose --name leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/orig.config.yaml 2024-04-26T18:05:42.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T18:05:42.725 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915 branch: wip-lusov-quiescer-fixes description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7674915' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer-fixes ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer-fixes sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9496 sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer-fixes suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKt2XrXN9sOIGeIXrA0FRAo33PJ4+N5alJPI025mlNybr2IIH122xsW+emsX1BiCi+miL2FZznRt2l5WxZT29JM= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEv0e7OwoLOnw4LxUSKbOh76Fryh8b8Kf8GTbnxUQecSnRnSd4Cp+tlaJ8pZtYLUmT66lT23lh+M0nvTYrIZv8M= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC4vY4i+oXqDIvsEzdk6zZW6C3rTc5FvDcGzGrZOUIJbNO2FQqm6g1Jecw/yWFRCSN/3folN98oQ6Gwm5wyph8o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_17:24:23 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T18:05:42.725 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa; will attempt to use it 2024-04-26T18:05:42.726 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks 2024-04-26T18:05:42.726 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T18:05:42.727 INFO:teuthology.task.internal:Checking packages... 2024-04-26T18:05:42.752 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1' 2024-04-26T18:05:42.753 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T18:05:42.753 INFO:teuthology.packaging:ref: None 2024-04-26T18:05:42.753 INFO:teuthology.packaging:tag: None 2024-04-26T18:05:42.753 INFO:teuthology.packaging:branch: wip-lusov-quiescer-fixes 2024-04-26T18:05:42.753 INFO:teuthology.packaging:sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:05:42.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer-fixes 2024-04-26T18:05:42.901 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3267.gbc94a2a9 2024-04-26T18:05:42.902 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T18:05:42.909 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T18:05:42.909 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T18:05:42.918 INFO:teuthology.task.internal:Saving configuration 2024-04-26T18:05:42.940 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T18:05:42.948 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T18:05:42.974 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:00:35.583375', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKt2XrXN9sOIGeIXrA0FRAo33PJ4+N5alJPI025mlNybr2IIH122xsW+emsX1BiCi+miL2FZznRt2l5WxZT29JM='} 2024-04-26T18:05:42.998 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:00:35.586939', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEv0e7OwoLOnw4LxUSKbOh76Fryh8b8Kf8GTbnxUQecSnRnSd4Cp+tlaJ8pZtYLUmT66lT23lh+M0nvTYrIZv8M='} 2024-04-26T18:05:43.018 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:00:35.585291', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC4vY4i+oXqDIvsEzdk6zZW6C3rTc5FvDcGzGrZOUIJbNO2FQqm6g1Jecw/yWFRCSN/3folN98oQ6Gwm5wyph8o='} 2024-04-26T18:05:43.018 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T18:05:43.025 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T18:05:43.025 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T18:05:43.026 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T18:05:43.026 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T18:05:43.129 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0cdf66e0d0>, signals=[15]) 2024-04-26T18:05:43.130 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T18:05:43.137 INFO:teuthology.task.internal:Opening connections... 2024-04-26T18:05:43.137 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:05:43.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:05:43.217 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T18:05:43.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:05:43.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-04-26T18:05:43.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:05:43.365 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T18:05:43.373 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-26T18:05:43.392 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-26T18:05:43.392 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:05:43.451 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:05:43.452 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:05:43.452 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:05:43.452 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:05:43.452 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:05:43.452 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-26T18:05:43.474 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-04-26T18:05:43.492 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-04-26T18:05:43.492 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-04-26T18:05:43.549 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:05:43.550 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:05:43.551 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:05:43.551 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-04-26T18:05:43.574 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-04-26T18:05:43.592 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-04-26T18:05:43.592 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-04-26T18:05:43.649 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-04-26T18:05:43.649 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-04-26T18:05:43.649 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-04-26T18:05:43.649 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:05:43.650 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:05:43.651 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-04-26T18:05:43.675 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T18:05:43.687 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T18:05:43.695 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T18:05:43.695 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:05:43.698 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:05:43.700 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:05:43.716 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T18:05:43.723 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T18:05:43.723 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:05:43.755 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:05:43.757 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:05:43.781 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T18:05:43.847 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T18:05:43.863 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T18:05:43.863 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-26T18:05:43.863 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-26T18:05:43.864 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:05:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:05:43.881 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-26T18:05:43.939 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-26T18:05:43.939 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-26T18:05:43.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-26T18:05:47.337 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-04-26T18:05:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-26T18:05:54.322 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-26T18:05:55.864 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-26T18:05:57.149 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-26T18:06:02.291 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-26T18:06:04.247 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T18:06:04.247 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T18:06:04.337 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T18:06:04.338 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-26T18:06:04.339 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T18:06:04.436 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-26T18:06:04.960 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 06:06:02 PM UTC. 2024-04-26T18:06:05.084 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T18:06:05.100 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-26T18:06:05.101 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-26T18:06:05.154 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-26T18:06:05.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-26T18:06:05.671 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 06:06:02 PM UTC. 2024-04-26T18:06:05.792 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T18:06:05.793 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-26T18:06:05.793 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.793 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-26T18:06:05.793 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-26T18:06:05.794 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-26T18:06:05.795 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-26T18:06:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.8 MB/s | 255 kB 00:00 2024-04-26T18:06:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:06:06.177 INFO:teuthology.orchestra.run.smithi037.stdout:Total 665 kB/s | 255 kB 00:00 2024-04-26T18:06:06.198 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-26T18:06:06.368 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-26T18:06:06.368 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-26T18:06:06.374 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-26T18:06:06.374 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-26T18:06:06.511 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-26T18:06:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T18:06:06.594 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:06:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:06:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:06:07.992 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T18:06:08.139 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T18:06:08.186 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T18:06:08.187 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T18:06:08.187 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T18:06:08.187 DEBUG:teuthology.parallel:result is None 2024-04-26T18:06:08.188 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T18:06:08.196 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T18:06:08.196 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:06:08.198 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:06:08.200 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:06:08.219 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T18:06:08.226 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T18:06:08.235 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T18:06:08.236 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:06:08.254 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:06:08.258 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:06:08.378 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T18:06:08.385 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T18:06:08.385 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:06:08.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:06:08.402 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:06:08.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:06:08.418 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:06:08.420 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:06:08.445 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:06:08.463 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.476 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.483 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.493 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.507 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.535 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:06:08.536 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T18:06:08.543 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T18:06:08.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:06:08.546 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:06:08.551 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:06:08.603 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T18:06:08.612 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T18:06:08.613 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:06:08.615 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:06:08.618 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:06:08.659 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:08.719 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:08.761 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:08.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:08.881 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:06:08.881 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:06:08.945 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:08.987 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:09.043 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:09.124 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:09.167 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:06:09.167 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:06:09.232 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:09.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:06:09.337 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:09.382 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:06:09.422 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:06:09.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:06:09.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-26T18:06:09.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-04-26T18:06:09.493 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-04-26T18:06:09.526 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:06:09.532 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:06:09.574 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:06:10.115 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T18:06:10.123 INFO:teuthology.task.internal:Starting timer... 2024-04-26T18:06:10.124 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T18:06:10.136 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T18:06:10.145 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T18:06:10.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-26T18:06:10.222 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-26T18:06:10.224 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-04-26T18:06:10.293 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-04-26T18:06:10.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-04-26T18:06:10.355 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-04-26T18:06:10.357 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T18:06:10.357 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-26T18:06:10.377 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-26T18:06:10.377 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-04-26T18:06:10.404 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-04-26T18:06:10.405 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-04-26T18:06:10.428 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-04-26T18:06:10.428 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-04-26T18:06:10.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:06:10.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:06:10.453 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:06:10.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:06:10.480 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:06:10.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:06:10.505 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T18:06:10.505 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:06:10.529 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:06:10.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:06:10.598 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T18:06:10.610 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T18:06:10.616 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T18:06:10.752 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T18:06:10.769 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T18:06:10.771 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi120.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T18:10:04.588 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-04-26T18:10:04.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T18:10:04.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:10:04.668 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-26T18:10:04.747 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T18:10:04.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-26T18:10:04.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:10:04.822 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-04-26T18:10:04.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-26T18:10:04.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-26T18:10:04.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:10:04.973 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-04-26T18:10:05.054 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-26T18:10:05.054 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T18:10:05.067 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T18:10:05.067 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:10:05.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:10:05.070 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:10:05.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:10:05.072 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:10:05.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:10:05.097 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:10:05.101 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:10:05.111 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:10:05.116 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:10:05.124 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:10:05.133 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-04-26T18:10:05.138 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:10:05.142 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-26T18:10:05.143 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-04-26T18:10:05.154 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-26T18:10:05.156 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:10:05.162 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-04-26T18:10:05.168 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:10:05.169 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:10:05.172 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-04-26T18:10:05.182 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:10:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:10:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:10:05.224 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.227 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.239 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:10:05.242 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:10:05.242 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T18:10:05.242 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.243 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.243 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.243 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:10:05.258 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:10:05.259 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-04-26T18:10:05.259 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.259 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.259 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.259 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:10:05.260 INFO:teuthology.run_tasks:Running task install... 2024-04-26T18:10:05.270 DEBUG:teuthology.task.install:project ceph 2024-04-26T18:10:05.270 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'}} 2024-04-26T18:10:05.270 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'} 2024-04-26T18:10:05.270 INFO:teuthology.task.install:Using flavor: default 2024-04-26T18:10:05.276 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T18:10:05.276 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T18:10:05.276 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:10:05.277 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:10:05.279 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:10:05.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:10:05.280 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:10:05.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:10:05.535 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:10:05.535 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:10:05.553 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:10:05.553 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:10:05.554 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:10:05.554 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:10:05.790 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:10:05.790 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:10:05.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:10:05.802 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:10:05.803 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:10:05.803 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:10:05.820 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:10:05.820 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:10:05.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:10:05.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:10:05.822 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:10:05.828 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:10:05.829 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:10:05.846 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:10:05.847 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:10:05.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:10:05.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:10:05.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:10:05.971 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:10:05.971 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:10:05.987 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:10:06.034 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-04-26T18:10:06.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-04-26T18:10:06.036 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-26T18:10:06.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-26T18:10:06.049 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-04-26T18:10:06.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-04-26T18:10:06.258 INFO:teuthology.orchestra.run.smithi170.stdout:48 files removed 2024-04-26T18:10:06.273 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-26T18:10:06.274 INFO:teuthology.orchestra.run.smithi120.stdout:48 files removed 2024-04-26T18:10:06.286 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:10:06.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:10:06.302 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:10:06.994 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 200 kB/s | 84 kB 00:00 2024-04-26T18:10:06.995 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 197 kB/s | 84 kB 00:00 2024-04-26T18:10:07.072 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 159 kB/s | 84 kB 00:00 2024-04-26T18:10:07.317 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-04-26T18:10:07.428 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-26T18:10:07.490 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T18:10:07.768 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-26T18:10:07.815 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-26T18:10:07.870 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-26T18:10:08.251 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-04-26T18:10:08.462 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T18:10:08.774 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-26T18:10:10.480 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T18:10:11.728 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-04-26T18:10:12.308 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-04-26T18:10:15.689 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-26T18:10:15.853 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T18:10:16.284 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-26T18:10:17.532 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-26T18:10:17.660 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-04-26T18:10:17.754 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-04-26T18:10:19.339 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T18:10:19.410 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T18:10:24.452 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T18:10:24.716 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T18:10:26.587 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.587 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.598 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:10:26.599 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.600 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:10:26.600 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:10:26.602 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:10:26.604 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:10:26.605 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:10:26.666 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:10:26.685 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:10:26.686 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:10:26.687 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:10:26.688 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:10:26.689 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:10:26.690 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:10:26.691 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:10:26.692 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:10:26.693 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:10:26.694 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:10:26.695 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:10:26.696 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout:Install 148 Packages 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 4 Packages 2024-04-26T18:10:26.697 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:10:26.698 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 222 M 2024-04-26T18:10:26.698 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-04-26T18:10:26.943 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.943 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.954 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:10:26.955 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:10:26.956 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:10:26.957 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:10:26.958 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:10:26.961 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:10:26.961 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:10:27.026 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:10:27.035 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:10:27.036 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:10:27.037 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:10:27.038 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:10:27.039 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:10:27.040 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:10:27.041 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:10:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:10:27.043 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:10:27.044 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:10:27.045 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:10:27.046 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:10:27.047 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-26T18:10:27.048 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-26T18:10:27.596 INFO:teuthology.orchestra.run.smithi170.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-26T18:10:27.954 INFO:teuthology.orchestra.run.smithi170.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 2.4 MB/s | 890 kB 00:00 2024-04-26T18:10:28.365 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-26T18:10:28.365 INFO:teuthology.orchestra.run.smithi170.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 2.1 MB/s | 145 kB 00:00 2024-04-26T18:10:28.559 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-26T18:10:28.596 INFO:teuthology.orchestra.run.smithi170.stdout:(4/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 4.1 MB/s | 2.4 MB 00:00 2024-04-26T18:10:28.710 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 965 kB/s | 145 kB 00:00 2024-04-26T18:10:28.780 INFO:teuthology.orchestra.run.smithi170.stdout:(5/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-26T18:10:28.802 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-04-26T18:10:29.005 INFO:teuthology.orchestra.run.smithi170.stdout:(6/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-26T18:10:29.110 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-04-26T18:10:29.213 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 21 MB 00:11 2024-04-26T18:10:29.669 INFO:teuthology.orchestra.run.smithi170.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-04-26T18:10:29.927 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-26T18:10:30.194 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-04-26T18:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout:(8/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el9 7.1 MB/s | 11 MB 00:01 2024-04-26T18:10:31.265 INFO:teuthology.orchestra.run.smithi170.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 231 kB/s | 25 kB 00:00 2024-04-26T18:10:31.556 INFO:teuthology.orchestra.run.smithi170.stdout:(10/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 6.6 MB/s | 17 MB 00:02 2024-04-26T18:10:31.657 INFO:teuthology.orchestra.run.smithi170.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 2.2 MB/s | 224 kB 00:00 2024-04-26T18:10:31.782 INFO:teuthology.orchestra.run.smithi170.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 216 kB/s | 27 kB 00:00 2024-04-26T18:10:31.840 INFO:teuthology.orchestra.run.smithi170.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 546 kB/s | 32 kB 00:00 2024-04-26T18:10:31.877 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-common-19.0.0-3267.gbc94a2a9.el9. 5.1 MB/s | 21 MB 00:04 2024-04-26T18:10:31.952 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 337 kB/s | 25 kB 00:00 2024-04-26T18:10:32.124 INFO:teuthology.orchestra.run.smithi170.stdout:(14/152): ceph-common-19.0.0-3267.gbc94a2a9.el9 4.4 MB/s | 21 MB 00:04 2024-04-26T18:10:32.158 INFO:teuthology.orchestra.run.smithi170.stdout:(15/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 2.3 MB/s | 746 kB 00:00 2024-04-26T18:10:32.208 INFO:teuthology.orchestra.run.smithi170.stdout:(16/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.9 MB/s | 163 kB 00:00 2024-04-26T18:10:32.250 INFO:teuthology.orchestra.run.smithi170.stdout:(17/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.3 MB/s | 127 kB 00:00 2024-04-26T18:10:32.367 INFO:teuthology.orchestra.run.smithi170.stdout:(18/152): libradosstriper1-19.0.0-3267.gbc94a2a 3.1 MB/s | 501 kB 00:00 2024-04-26T18:10:32.467 INFO:teuthology.orchestra.run.smithi170.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 451 kB/s | 45 kB 00:00 2024-04-26T18:10:32.584 INFO:teuthology.orchestra.run.smithi170.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.1 MB/s | 135 kB 00:00 2024-04-26T18:10:32.693 INFO:teuthology.orchestra.run.smithi170.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 1.5 MB/s | 162 kB 00:00 2024-04-26T18:10:32.728 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el 4.2 MB/s | 11 MB 00:02 2024-04-26T18:10:32.794 INFO:teuthology.orchestra.run.smithi170.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 3.1 MB/s | 320 kB 00:00 2024-04-26T18:10:32.854 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 1.8 MB/s | 224 kB 00:00 2024-04-26T18:10:32.919 INFO:teuthology.orchestra.run.smithi170.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T18:10:32.929 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 361 kB/s | 27 kB 00:00 2024-04-26T18:10:33.002 INFO:teuthology.orchestra.run.smithi170.stdout:(24/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-04-26T18:10:33.015 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 381 kB/s | 32 kB 00:00 2024-04-26T18:10:33.030 INFO:teuthology.orchestra.run.smithi170.stdout:(25/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 916 kB/s | 99 kB 00:00 2024-04-26T18:10:33.069 INFO:teuthology.orchestra.run.smithi170.stdout:(26/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T18:10:33.172 INFO:teuthology.orchestra.run.smithi170.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-26T18:10:33.247 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 3.1 MB/s | 746 kB 00:00 2024-04-26T18:10:33.257 INFO:teuthology.orchestra.run.smithi170.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 372 kB/s | 34 kB 00:00 2024-04-26T18:10:33.305 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 2.7 MB/s | 163 kB 00:00 2024-04-26T18:10:33.345 INFO:teuthology.orchestra.run.smithi170.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 1.5 MB/s | 142 kB 00:00 2024-04-26T18:10:33.385 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.6 MB/s | 127 kB 00:00 2024-04-26T18:10:33.696 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 1.5 MB/s | 501 kB 00:00 2024-04-26T18:10:33.730 INFO:teuthology.orchestra.run.smithi170.stdout:(30/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 4.4 MB/s | 3.1 MB 00:00 2024-04-26T18:10:33.871 INFO:teuthology.orchestra.run.smithi170.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 6.7 MB/s | 3.5 MB 00:00 2024-04-26T18:10:33.963 INFO:teuthology.orchestra.run.smithi170.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 2.7 MB/s | 249 kB 00:00 2024-04-26T18:10:34.046 INFO:teuthology.orchestra.run.smithi170.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 593 kB/s | 49 kB 00:00 2024-04-26T18:10:34.121 INFO:teuthology.orchestra.run.smithi170.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 219 kB/s | 16 kB 00:00 2024-04-26T18:10:34.222 INFO:teuthology.orchestra.run.smithi170.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.7 MB/s | 273 kB 00:00 2024-04-26T18:10:34.310 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T18:10:34.389 INFO:teuthology.orchestra.run.smithi170.stdout:(36/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-26T18:10:34.489 INFO:teuthology.orchestra.run.smithi170.stdout:(37/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 182 kB/s | 18 kB 00:00 2024-04-26T18:10:34.556 INFO:teuthology.orchestra.run.smithi170.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-26T18:10:34.664 INFO:teuthology.orchestra.run.smithi170.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-26T18:10:34.740 INFO:teuthology.orchestra.run.smithi170.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-26T18:10:34.815 INFO:teuthology.orchestra.run.smithi170.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-26T18:10:34.949 INFO:teuthology.orchestra.run.smithi170.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-26T18:10:35.016 INFO:teuthology.orchestra.run.smithi170.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T18:10:35.074 INFO:teuthology.orchestra.run.smithi170.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T18:10:35.113 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 3.8 MB/s | 5.4 MB 00:01 2024-04-26T18:10:35.158 INFO:teuthology.orchestra.run.smithi170.stdout:(45/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-26T18:10:35.164 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 908 kB/s | 45 kB 00:00 2024-04-26T18:10:35.217 INFO:teuthology.orchestra.run.smithi170.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 880 kB/s | 125 kB 00:00 2024-04-26T18:10:35.222 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 2.3 MB/s | 135 kB 00:00 2024-04-26T18:10:35.259 INFO:teuthology.orchestra.run.smithi170.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T18:10:35.289 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 2.4 MB/s | 162 kB 00:00 2024-04-26T18:10:35.381 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 3.4 MB/s | 320 kB 00:00 2024-04-26T18:10:35.465 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T18:10:35.484 INFO:teuthology.orchestra.run.smithi170.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-04-26T18:10:35.515 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.9 MB/s | 99 kB 00:00 2024-04-26T18:10:35.542 INFO:teuthology.orchestra.run.smithi170.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T18:10:35.568 INFO:teuthology.orchestra.run.smithi170.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-04-26T18:10:35.593 INFO:teuthology.orchestra.run.smithi170.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-04-26T18:10:35.607 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 931 kB/s | 85 kB 00:00 2024-04-26T18:10:35.643 INFO:teuthology.orchestra.run.smithi170.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-26T18:10:35.669 INFO:teuthology.orchestra.run.smithi170.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-26T18:10:35.720 INFO:teuthology.orchestra.run.smithi170.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-26T18:10:35.769 INFO:teuthology.orchestra.run.smithi170.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-26T18:10:35.819 INFO:teuthology.orchestra.run.smithi170.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T18:10:35.870 INFO:teuthology.orchestra.run.smithi170.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-26T18:10:36.133 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 5.9 MB/s | 3.1 MB 00:00 2024-04-26T18:10:36.183 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-26T18:10:36.233 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 684 kB/s | 34 kB 00:00 2024-04-26T18:10:36.292 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 2.4 MB/s | 142 kB 00:00 2024-04-26T18:10:36.387 INFO:teuthology.orchestra.run.smithi170.stdout:(58/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 9.6 MB/s | 49 MB 00:05 2024-04-26T18:10:36.412 INFO:teuthology.orchestra.run.smithi170.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.2 MB/s | 662 kB 00:00 2024-04-26T18:10:36.456 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:36.457 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:10:36.462 INFO:teuthology.orchestra.run.smithi170.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-04-26T18:10:36.467 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:10:36.468 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:10:36.469 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:10:36.471 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:10:36.473 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:10:36.474 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:10:36.487 INFO:teuthology.orchestra.run.smithi170.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-26T18:10:36.513 INFO:teuthology.orchestra.run.smithi170.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-26T18:10:36.534 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:10:36.550 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:10:36.551 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:10:36.552 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:10:36.553 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:10:36.554 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:10:36.555 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:10:36.556 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:10:36.557 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:10:36.558 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:10:36.559 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:10:36.560 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:10:36.561 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:10:36.562 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:10:36.563 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:10:36.564 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:10:36.565 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 4 Packages 2024-04-26T18:10:36.566 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:10:36.567 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 222 M 2024-04-26T18:10:36.567 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-04-26T18:10:36.640 INFO:teuthology.orchestra.run.smithi170.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-26T18:10:36.666 INFO:teuthology.orchestra.run.smithi170.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-04-26T18:10:36.699 INFO:teuthology.orchestra.run.smithi170.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-26T18:10:36.750 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 7.7 MB/s | 3.5 MB 00:00 2024-04-26T18:10:36.841 INFO:teuthology.orchestra.run.smithi170.stdout:(66/152): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-26T18:10:37.176 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 2.3 MB/s | 17 MB 00:07 2024-04-26T18:10:37.233 INFO:teuthology.orchestra.run.smithi170.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-04-26T18:10:37.301 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 2.0 MB/s | 249 kB 00:00 2024-04-26T18:10:37.376 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 661 kB/s | 49 kB 00:00 2024-04-26T18:10:37.415 INFO:teuthology.orchestra.run.smithi120.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-26T18:10:37.417 INFO:teuthology.orchestra.run.smithi170.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.3 MB/s | 4.7 MB 00:00 2024-04-26T18:10:37.443 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 247 kB/s | 16 kB 00:00 2024-04-26T18:10:37.475 INFO:teuthology.orchestra.run.smithi170.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-04-26T18:10:37.501 INFO:teuthology.orchestra.run.smithi170.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T18:10:37.569 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T18:10:37.692 INFO:teuthology.orchestra.run.smithi170.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-26T18:10:37.768 INFO:teuthology.orchestra.run.smithi170.stdout:(72/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-04-26T18:10:37.793 INFO:teuthology.orchestra.run.smithi170.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 872 kB/s | 87 kB 00:00 2024-04-26T18:10:37.852 INFO:teuthology.orchestra.run.smithi170.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T18:10:37.910 INFO:teuthology.orchestra.run.smithi170.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-26T18:10:37.960 INFO:teuthology.orchestra.run.smithi170.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-26T18:10:37.982 INFO:teuthology.orchestra.run.smithi120.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-26T18:10:38.011 INFO:teuthology.orchestra.run.smithi170.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-26T18:10:38.069 INFO:teuthology.orchestra.run.smithi170.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T18:10:38.098 INFO:teuthology.orchestra.run.smithi120.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 1.2 MB/s | 145 kB 00:00 2024-04-26T18:10:38.120 INFO:teuthology.orchestra.run.smithi170.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-26T18:10:38.127 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 8.0 MB/s | 49 MB 00:06 2024-04-26T18:10:38.161 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-26T18:10:38.186 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 312 kB/s | 18 kB 00:00 2024-04-26T18:10:38.278 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-26T18:10:38.328 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-04-26T18:10:38.370 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-26T18:10:38.420 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T18:10:38.448 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-26T18:10:38.506 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-26T18:10:38.540 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-26T18:10:38.582 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-04-26T18:10:38.607 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-26T18:10:38.657 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-26T18:10:38.703 INFO:teuthology.orchestra.run.smithi170.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-04-26T18:10:38.724 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-04-26T18:10:38.732 INFO:teuthology.orchestra.run.smithi120.stdout:(4/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-04-26T18:10:38.750 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-26T18:10:38.770 INFO:teuthology.orchestra.run.smithi170.stdout:(81/152): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-04-26T18:10:38.792 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-04-26T18:10:38.795 INFO:teuthology.orchestra.run.smithi170.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-26T18:10:38.817 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-04-26T18:10:38.845 INFO:teuthology.orchestra.run.smithi170.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T18:10:38.858 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-04-26T18:10:38.884 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-26T18:10:38.896 INFO:teuthology.orchestra.run.smithi170.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-26T18:10:38.926 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-26T18:10:38.938 INFO:teuthology.orchestra.run.smithi170.stdout:(85/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-26T18:10:38.941 INFO:teuthology.orchestra.run.smithi120.stdout:(5/152): ceph-common-19.0.0-3267.gbc94a2a9.el9. 12 MB/s | 21 MB 00:01 2024-04-26T18:10:38.951 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-26T18:10:38.963 INFO:teuthology.orchestra.run.smithi170.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-26T18:10:39.016 INFO:teuthology.orchestra.run.smithi120.stdout:(6/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-04-26T18:10:39.018 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 681 kB/s | 45 kB 00:00 2024-04-26T18:10:39.021 INFO:teuthology.orchestra.run.smithi170.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-26T18:10:39.043 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T18:10:39.047 INFO:teuthology.orchestra.run.smithi170.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-26T18:10:39.066 INFO:teuthology.orchestra.run.smithi120.stdout:(7/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-26T18:10:39.072 INFO:teuthology.orchestra.run.smithi170.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T18:10:39.098 INFO:teuthology.orchestra.run.smithi170.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T18:10:39.102 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T18:10:39.144 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 614 kB/s | 25 kB 00:00 2024-04-26T18:10:39.148 INFO:teuthology.orchestra.run.smithi170.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-26T18:10:39.194 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-26T18:10:39.207 INFO:teuthology.orchestra.run.smithi170.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-04-26T18:10:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-04-26T18:10:39.245 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-04-26T18:10:39.303 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-26T18:10:39.315 INFO:teuthology.orchestra.run.smithi170.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-26T18:10:39.316 INFO:teuthology.orchestra.run.smithi120.stdout:(8/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-26T18:10:39.320 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-26T18:10:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-26T18:10:39.358 INFO:teuthology.orchestra.run.smithi120.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 606 kB/s | 25 kB 00:00 2024-04-26T18:10:39.374 INFO:teuthology.orchestra.run.smithi170.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-04-26T18:10:39.433 INFO:teuthology.orchestra.run.smithi170.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-26T18:10:39.537 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T18:10:39.575 INFO:teuthology.orchestra.run.smithi170.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-04-26T18:10:39.633 INFO:teuthology.orchestra.run.smithi170.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 992 kB/s | 58 kB 00:00 2024-04-26T18:10:39.638 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-26T18:10:39.704 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T18:10:39.709 INFO:teuthology.orchestra.run.smithi170.stdout:(98/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-04-26T18:10:39.734 INFO:teuthology.orchestra.run.smithi170.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 933 kB/s | 94 kB 00:00 2024-04-26T18:10:39.759 INFO:teuthology.orchestra.run.smithi170.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T18:10:39.771 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-26T18:10:39.793 INFO:teuthology.orchestra.run.smithi170.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-04-26T18:10:39.805 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-04-26T18:10:39.818 INFO:teuthology.orchestra.run.smithi170.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-04-26T18:10:39.872 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T18:10:39.981 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-26T18:10:40.056 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T18:10:40.127 INFO:teuthology.orchestra.run.smithi170.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-26T18:10:40.156 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-26T18:10:40.206 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-26T18:10:40.232 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-04-26T18:10:40.274 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-26T18:10:40.316 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-26T18:10:40.341 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-26T18:10:40.441 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-26T18:10:40.492 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-26T18:10:40.542 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T18:10:40.567 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T18:10:40.577 INFO:teuthology.orchestra.run.smithi170.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 723 kB/s | 548 kB 00:00 2024-04-26T18:10:40.617 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-04-26T18:10:40.668 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T18:10:40.694 INFO:teuthology.orchestra.run.smithi170.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-26T18:10:40.734 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-26T18:10:40.762 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-26T18:10:40.810 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-04-26T18:10:40.819 INFO:teuthology.orchestra.run.smithi170.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 446 kB/s | 309 kB 00:00 2024-04-26T18:10:40.835 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T18:10:40.861 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-04-26T18:10:40.911 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-26T18:10:40.937 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-26T18:10:40.937 INFO:teuthology.orchestra.run.smithi170.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-26T18:10:40.986 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 722 kB/s | 36 kB 00:00 2024-04-26T18:10:41.012 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-26T18:10:41.062 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-26T18:10:41.111 INFO:teuthology.orchestra.run.smithi170.stdout:(108/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.0 MB/s | 16 MB 00:02 2024-04-26T18:10:41.112 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-26T18:10:41.137 INFO:teuthology.orchestra.run.smithi170.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 242 kB/s | 49 kB 00:00 2024-04-26T18:10:41.137 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-04-26T18:10:41.196 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-26T18:10:41.204 INFO:teuthology.orchestra.run.smithi170.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-04-26T18:10:41.213 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 849 kB/s | 64 kB 00:00 2024-04-26T18:10:41.263 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-04-26T18:10:41.279 INFO:teuthology.orchestra.run.smithi170.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T18:10:41.288 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-26T18:10:41.354 INFO:teuthology.orchestra.run.smithi170.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 839 kB 00:00 2024-04-26T18:10:41.358 INFO:teuthology.orchestra.run.smithi120.stdout:(10/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 7.2 MB/s | 17 MB 00:02 2024-04-26T18:10:41.455 INFO:teuthology.orchestra.run.smithi170.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-26T18:10:41.511 INFO:teuthology.orchestra.run.smithi120.stdout:(11/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el 4.3 MB/s | 11 MB 00:02 2024-04-26T18:10:41.536 INFO:teuthology.orchestra.run.smithi120.stdout:(12/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 1.2 MB/s | 224 kB 00:00 2024-04-26T18:10:41.564 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-26T18:10:41.592 INFO:teuthology.orchestra.run.smithi170.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-26T18:10:41.613 INFO:teuthology.orchestra.run.smithi170.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 203 kB/s | 32 kB 00:00 2024-04-26T18:10:41.620 INFO:teuthology.orchestra.run.smithi120.stdout:(13/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 249 kB/s | 27 kB 00:00 2024-04-26T18:10:41.637 INFO:teuthology.orchestra.run.smithi120.stdout:(14/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 316 kB/s | 32 kB 00:00 2024-04-26T18:10:41.638 INFO:teuthology.orchestra.run.smithi170.stdout:(116/152): libunwind-1.6.2-1.el9.x86_64.rpm 128 kB/s | 67 kB 00:00 2024-04-26T18:10:41.689 INFO:teuthology.orchestra.run.smithi170.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 156 kB/s | 15 kB 00:00 2024-04-26T18:10:41.696 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-26T18:10:41.714 INFO:teuthology.orchestra.run.smithi170.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-26T18:10:41.729 INFO:teuthology.orchestra.run.smithi120.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.7 MB/s | 163 kB 00:00 2024-04-26T18:10:41.782 INFO:teuthology.orchestra.run.smithi120.stdout:(16/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 4.6 MB/s | 746 kB 00:00 2024-04-26T18:10:41.813 INFO:teuthology.orchestra.run.smithi120.stdout:(17/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.5 MB/s | 127 kB 00:00 2024-04-26T18:10:41.823 INFO:teuthology.orchestra.run.smithi170.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-26T18:10:41.848 INFO:teuthology.orchestra.run.smithi170.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-26T18:10:41.940 INFO:teuthology.orchestra.run.smithi170.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-26T18:10:41.946 INFO:teuthology.orchestra.run.smithi120.stdout:(18/152): libradosstriper1-19.0.0-3267.gbc94a2a 2.9 MB/s | 501 kB 00:00 2024-04-26T18:10:41.957 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 821 kB/s | 548 kB 00:00 2024-04-26T18:10:41.974 INFO:teuthology.orchestra.run.smithi170.stdout:(122/152): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-04-26T18:10:41.982 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 76 kB/s | 19 kB 00:00 2024-04-26T18:10:42.024 INFO:teuthology.orchestra.run.smithi170.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T18:10:42.038 INFO:teuthology.orchestra.run.smithi120.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 493 kB/s | 45 kB 00:00 2024-04-26T18:10:42.066 INFO:teuthology.orchestra.run.smithi170.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-26T18:10:42.074 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 608 kB/s | 309 kB 00:00 2024-04-26T18:10:42.091 INFO:teuthology.orchestra.run.smithi170.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-26T18:10:42.099 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-26T18:10:42.133 INFO:teuthology.orchestra.run.smithi170.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-04-26T18:10:42.149 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-26T18:10:42.155 INFO:teuthology.orchestra.run.smithi120.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.1 MB/s | 135 kB 00:00 2024-04-26T18:10:42.175 INFO:teuthology.orchestra.run.smithi170.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-26T18:10:42.201 INFO:teuthology.orchestra.run.smithi170.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-26T18:10:42.242 INFO:teuthology.orchestra.run.smithi170.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-26T18:10:42.249 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T18:10:42.275 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 383 kB/s | 67 kB 00:00 2024-04-26T18:10:42.300 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T18:10:42.318 INFO:teuthology.orchestra.run.smithi170.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-26T18:10:42.342 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-04-26T18:10:42.351 INFO:teuthology.orchestra.run.smithi170.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-26T18:10:42.385 INFO:teuthology.orchestra.run.smithi170.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-26T18:10:42.392 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-04-26T18:10:42.417 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-26T18:10:42.427 INFO:teuthology.orchestra.run.smithi170.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-26T18:10:42.460 INFO:teuthology.orchestra.run.smithi170.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-26T18:10:42.486 INFO:teuthology.orchestra.run.smithi170.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-26T18:10:42.501 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-26T18:10:42.526 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-04-26T18:10:42.544 INFO:teuthology.orchestra.run.smithi170.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-04-26T18:10:42.551 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-26T18:10:42.561 INFO:teuthology.orchestra.run.smithi170.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-26T18:10:42.587 INFO:teuthology.orchestra.run.smithi170.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-26T18:10:42.606 INFO:teuthology.orchestra.run.smithi120.stdout:(21/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 15 MB/s | 49 MB 00:03 2024-04-26T18:10:42.619 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T18:10:42.631 INFO:teuthology.orchestra.run.smithi120.stdout:(22/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 340 kB/s | 162 kB 00:00 2024-04-26T18:10:42.644 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-26T18:10:42.662 INFO:teuthology.orchestra.run.smithi170.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-26T18:10:42.669 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-26T18:10:42.687 INFO:teuthology.orchestra.run.smithi170.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-26T18:10:42.703 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-26T18:10:42.704 INFO:teuthology.orchestra.run.smithi170.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-04-26T18:10:42.723 INFO:teuthology.orchestra.run.smithi120.stdout:(23/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-04-26T18:10:42.748 INFO:teuthology.orchestra.run.smithi120.stdout:(24/152): python3-rados-19.0.0-3267.gbc94a2a9.e 2.2 MB/s | 320 kB 00:00 2024-04-26T18:10:42.763 INFO:teuthology.orchestra.run.smithi170.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-26T18:10:42.769 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-26T18:10:42.773 INFO:teuthology.orchestra.run.smithi120.stdout:(25/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 2.1 MB/s | 298 kB 00:00 2024-04-26T18:10:42.795 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-26T18:10:42.799 INFO:teuthology.orchestra.run.smithi120.stdout:(26/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-26T18:10:42.804 INFO:teuthology.orchestra.run.smithi170.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-04-26T18:10:42.820 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 960 kB/s | 144 kB 00:00 2024-04-26T18:10:42.824 INFO:teuthology.orchestra.run.smithi120.stdout:(27/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 988 kB/s | 99 kB 00:00 2024-04-26T18:10:42.830 INFO:teuthology.orchestra.run.smithi170.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-26T18:10:42.845 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-26T18:10:42.866 INFO:teuthology.orchestra.run.smithi120.stdout:(28/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-26T18:10:42.871 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-26T18:10:42.880 INFO:teuthology.orchestra.run.smithi170.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-26T18:10:42.891 INFO:teuthology.orchestra.run.smithi120.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3267.g 511 kB/s | 34 kB 00:00 2024-04-26T18:10:42.913 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-26T18:10:42.914 INFO:teuthology.orchestra.run.smithi170.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-26T18:10:42.933 INFO:teuthology.orchestra.run.smithi120.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 2.1 MB/s | 142 kB 00:00 2024-04-26T18:10:42.938 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-26T18:10:42.964 INFO:teuthology.orchestra.run.smithi170.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T18:10:42.988 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-26T18:10:43.005 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-26T18:10:43.022 INFO:teuthology.orchestra.run.smithi170.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T18:10:43.039 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T18:10:43.072 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-04-26T18:10:43.098 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T18:10:43.123 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-04-26T18:10:43.131 INFO:teuthology.orchestra.run.smithi170.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T18:10:43.149 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T18:10:43.167 INFO:teuthology.orchestra.run.smithi120.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 13 MB/s | 3.5 MB 00:00 2024-04-26T18:10:43.174 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-26T18:10:43.199 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-26T18:10:43.233 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T18:10:43.251 INFO:teuthology.orchestra.run.smithi120.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 2.9 MB/s | 249 kB 00:00 2024-04-26T18:10:43.258 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-26T18:10:43.265 INFO:teuthology.orchestra.run.smithi170.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-26T18:10:43.283 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-26T18:10:43.301 INFO:teuthology.orchestra.run.smithi120.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 988 kB/s | 49 kB 00:00 2024-04-26T18:10:43.308 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-26T18:10:43.334 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T18:10:43.360 INFO:teuthology.orchestra.run.smithi120.stdout:(34/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 5.3 MB/s | 3.1 MB 00:00 2024-04-26T18:10:43.373 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 996 kB/s | 90 kB 00:00 2024-04-26T18:10:43.385 INFO:teuthology.orchestra.run.smithi120.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3267.gb 197 kB/s | 16 kB 00:00 2024-04-26T18:10:43.401 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-26T18:10:43.426 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-26T18:10:43.453 INFO:teuthology.orchestra.run.smithi120.stdout:(36/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T18:10:43.459 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T18:10:43.495 INFO:teuthology.orchestra.run.smithi120.stdout:(37/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 6.8 MB/s | 762 kB 00:00 2024-04-26T18:10:43.510 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-26T18:10:43.553 INFO:teuthology.orchestra.run.smithi120.stdout:(38/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 182 kB/s | 18 kB 00:00 2024-04-26T18:10:43.627 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T18:10:43.637 INFO:teuthology.orchestra.run.smithi120.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 566 kB/s | 80 kB 00:00 2024-04-26T18:10:43.640 INFO:teuthology.orchestra.run.smithi170.stdout:(151/152): librados2-19.0.0-3267.gbc94a2a9.el9. 4.7 MB/s | 3.4 MB 00:00 2024-04-26T18:10:43.762 INFO:teuthology.orchestra.run.smithi120.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-26T18:10:43.788 INFO:teuthology.orchestra.run.smithi120.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-26T18:10:43.790 INFO:teuthology.orchestra.run.smithi170.stdout:(152/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-26T18:10:43.797 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:10:43.797 INFO:teuthology.orchestra.run.smithi170.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T18:10:43.846 INFO:teuthology.orchestra.run.smithi120.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-26T18:10:43.886 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-04-26T18:10:43.905 INFO:teuthology.orchestra.run.smithi120.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T18:10:43.963 INFO:teuthology.orchestra.run.smithi120.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T18:10:43.988 INFO:teuthology.orchestra.run.smithi120.stdout:(45/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-26T18:10:44.011 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-26T18:10:44.014 INFO:teuthology.orchestra.run.smithi120.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T18:10:44.048 INFO:teuthology.orchestra.run.smithi120.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-04-26T18:10:44.073 INFO:teuthology.orchestra.run.smithi120.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T18:10:44.123 INFO:teuthology.orchestra.run.smithi120.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T18:10:44.173 INFO:teuthology.orchestra.run.smithi120.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-26T18:10:44.332 INFO:teuthology.orchestra.run.smithi120.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-04-26T18:10:44.357 INFO:teuthology.orchestra.run.smithi120.stdout:(52/152): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-04-26T18:10:44.383 INFO:teuthology.orchestra.run.smithi120.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-04-26T18:10:44.433 INFO:teuthology.orchestra.run.smithi120.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-26T18:10:44.475 INFO:teuthology.orchestra.run.smithi120.stdout:(55/152): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-04-26T18:10:44.500 INFO:teuthology.orchestra.run.smithi120.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-26T18:10:44.511 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): librados2-19.0.0-3267.gbc94a2a9.el9. 3.1 MB/s | 3.4 MB 00:01 2024-04-26T18:10:44.518 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:10:44.518 INFO:teuthology.orchestra.run.smithi037.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T18:10:44.550 INFO:teuthology.orchestra.run.smithi120.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-26T18:10:44.584 INFO:teuthology.orchestra.run.smithi120.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T18:10:44.760 INFO:teuthology.orchestra.run.smithi120.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-26T18:10:44.785 INFO:teuthology.orchestra.run.smithi120.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-04-26T18:10:44.810 INFO:teuthology.orchestra.run.smithi120.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-26T18:10:44.836 INFO:teuthology.orchestra.run.smithi120.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-04-26T18:10:44.919 INFO:teuthology.orchestra.run.smithi120.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T18:10:44.920 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-04-26T18:10:44.945 INFO:teuthology.orchestra.run.smithi120.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-26T18:10:44.970 INFO:teuthology.orchestra.run.smithi120.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-04-26T18:10:45.063 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-04-26T18:10:45.063 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-04-26T18:10:45.337 INFO:teuthology.orchestra.run.smithi120.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-04-26T18:10:45.652 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-26T18:10:45.813 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-26T18:10:45.813 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-26T18:10:46.071 INFO:teuthology.orchestra.run.smithi120.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 4.2 MB/s | 4.7 MB 00:01 2024-04-26T18:10:46.112 INFO:teuthology.orchestra.run.smithi120.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-04-26T18:10:46.196 INFO:teuthology.orchestra.run.smithi120.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T18:10:46.313 INFO:teuthology.orchestra.run.smithi120.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-04-26T18:10:46.538 INFO:teuthology.orchestra.run.smithi120.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-04-26T18:10:46.597 INFO:teuthology.orchestra.run.smithi120.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-04-26T18:10:46.622 INFO:teuthology.orchestra.run.smithi120.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-26T18:10:46.730 INFO:teuthology.orchestra.run.smithi120.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-26T18:10:46.869 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-04-26T18:10:46.870 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-04-26T18:10:46.964 INFO:teuthology.orchestra.run.smithi120.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-04-26T18:10:47.040 INFO:teuthology.orchestra.run.smithi120.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-26T18:10:47.123 INFO:teuthology.orchestra.run.smithi120.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-04-26T18:10:47.390 INFO:teuthology.orchestra.run.smithi120.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 645 kB/s | 172 kB 00:00 2024-04-26T18:10:47.449 INFO:teuthology.orchestra.run.smithi120.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-04-26T18:10:47.694 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-26T18:10:47.694 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-26T18:10:47.866 INFO:teuthology.orchestra.run.smithi120.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-04-26T18:10:48.100 INFO:teuthology.orchestra.run.smithi120.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-04-26T18:10:48.392 INFO:teuthology.orchestra.run.smithi120.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-04-26T18:10:48.425 INFO:teuthology.orchestra.run.smithi120.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 683 kB/s | 222 kB 00:00 2024-04-26T18:10:48.459 INFO:teuthology.orchestra.run.smithi120.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-26T18:10:48.484 INFO:teuthology.orchestra.run.smithi120.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-04-26T18:10:48.559 INFO:teuthology.orchestra.run.smithi120.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T18:10:48.584 INFO:teuthology.orchestra.run.smithi120.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-26T18:10:48.626 INFO:teuthology.orchestra.run.smithi120.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-26T18:10:48.652 INFO:teuthology.orchestra.run.smithi120.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-26T18:10:48.686 INFO:teuthology.orchestra.run.smithi120.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 910 kB/s | 54 kB 00:00 2024-04-26T18:10:48.744 INFO:teuthology.orchestra.run.smithi120.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-04-26T18:10:48.803 INFO:teuthology.orchestra.run.smithi120.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-26T18:10:48.928 INFO:teuthology.orchestra.run.smithi120.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-26T18:10:48.979 INFO:teuthology.orchestra.run.smithi120.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-26T18:10:49.029 INFO:teuthology.orchestra.run.smithi120.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-26T18:10:49.113 INFO:teuthology.orchestra.run.smithi120.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-26T18:10:49.163 INFO:teuthology.orchestra.run.smithi120.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T18:10:49.222 INFO:teuthology.orchestra.run.smithi120.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T18:10:49.272 INFO:teuthology.orchestra.run.smithi120.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T18:10:49.322 INFO:teuthology.orchestra.run.smithi120.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-26T18:10:49.372 INFO:teuthology.orchestra.run.smithi120.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-26T18:10:49.548 INFO:teuthology.orchestra.run.smithi120.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-26T18:10:49.648 INFO:teuthology.orchestra.run.smithi120.stdout:(103/152): python3-numpy-1.20.1-5.el9.x86_64.rp 2.3 MB/s | 5.1 MB 00:02 2024-04-26T18:10:50.190 INFO:teuthology.orchestra.run.smithi120.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 854 kB/s | 548 kB 00:00 2024-04-26T18:10:50.215 INFO:teuthology.orchestra.run.smithi120.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 545 kB/s | 309 kB 00:00 2024-04-26T18:10:50.299 INFO:teuthology.orchestra.run.smithi120.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-26T18:10:50.399 INFO:teuthology.orchestra.run.smithi120.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-26T18:10:50.524 INFO:teuthology.orchestra.run.smithi120.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-26T18:10:50.633 INFO:teuthology.orchestra.run.smithi120.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-26T18:10:50.700 INFO:teuthology.orchestra.run.smithi120.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-26T18:10:50.750 INFO:teuthology.orchestra.run.smithi120.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-26T18:10:50.826 INFO:teuthology.orchestra.run.smithi120.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-04-26T18:10:50.918 INFO:teuthology.orchestra.run.smithi120.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-26T18:10:50.943 INFO:teuthology.orchestra.run.smithi120.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-26T18:10:51.018 INFO:teuthology.orchestra.run.smithi120.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 320 kB/s | 32 kB 00:00 2024-04-26T18:10:51.044 INFO:teuthology.orchestra.run.smithi120.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-26T18:10:51.128 INFO:teuthology.orchestra.run.smithi120.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-26T18:10:51.153 INFO:teuthology.orchestra.run.smithi120.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-26T18:10:51.253 INFO:teuthology.orchestra.run.smithi120.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-26T18:10:51.279 INFO:teuthology.orchestra.run.smithi120.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-04-26T18:10:51.370 INFO:teuthology.orchestra.run.smithi120.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-26T18:10:51.421 INFO:teuthology.orchestra.run.smithi120.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-26T18:10:51.480 INFO:teuthology.orchestra.run.smithi120.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-26T18:10:51.513 INFO:teuthology.orchestra.run.smithi120.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-26T18:10:51.572 INFO:teuthology.orchestra.run.smithi120.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-26T18:10:51.606 INFO:teuthology.orchestra.run.smithi120.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-26T18:10:51.673 INFO:teuthology.orchestra.run.smithi120.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-26T18:10:51.698 INFO:teuthology.orchestra.run.smithi120.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-26T18:10:51.798 INFO:teuthology.orchestra.run.smithi120.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-26T18:10:51.857 INFO:teuthology.orchestra.run.smithi120.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-26T18:10:51.899 INFO:teuthology.orchestra.run.smithi120.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-26T18:10:51.957 INFO:teuthology.orchestra.run.smithi120.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-26T18:10:51.999 INFO:teuthology.orchestra.run.smithi120.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T18:10:52.058 INFO:teuthology.orchestra.run.smithi120.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-26T18:10:52.100 INFO:teuthology.orchestra.run.smithi120.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-26T18:10:52.159 INFO:teuthology.orchestra.run.smithi120.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-26T18:10:52.200 INFO:teuthology.orchestra.run.smithi120.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-26T18:10:52.259 INFO:teuthology.orchestra.run.smithi120.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-26T18:10:52.301 INFO:teuthology.orchestra.run.smithi120.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-26T18:10:52.360 INFO:teuthology.orchestra.run.smithi120.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-26T18:10:52.402 INFO:teuthology.orchestra.run.smithi120.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-26T18:10:52.469 INFO:teuthology.orchestra.run.smithi120.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T18:10:52.503 INFO:teuthology.orchestra.run.smithi120.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-04-26T18:10:52.578 INFO:teuthology.orchestra.run.smithi120.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T18:10:52.603 INFO:teuthology.orchestra.run.smithi120.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-26T18:10:52.774 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-04-26T18:10:52.779 INFO:teuthology.orchestra.run.smithi120.stdout:(146/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:04 2024-04-26T18:10:52.815 INFO:teuthology.orchestra.run.smithi120.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 810 kB/s | 192 kB 00:00 2024-04-26T18:10:52.854 INFO:teuthology.orchestra.run.smithi120.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-04-26T18:10:52.877 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:10:52.915 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:10:53.044 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:10:53.162 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:10:53.187 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:10:53.230 INFO:teuthology.orchestra.run.smithi120.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.6 MB/s | 1.3 MB 00:00 2024-04-26T18:10:53.230 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:10:53.313 INFO:teuthology.orchestra.run.smithi120.stdout:(150/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-04-26T18:10:53.397 INFO:teuthology.orchestra.run.smithi120.stdout:(151/152): librados2-19.0.0-3267.gbc94a2a9.el9. 5.5 MB/s | 3.4 MB 00:00 2024-04-26T18:10:53.559 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:10:53.655 INFO:teuthology.orchestra.run.smithi120.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 2.4 MB 00:00 2024-04-26T18:10:53.658 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:10:53.659 INFO:teuthology.orchestra.run.smithi120.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T18:10:53.671 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:10:53.689 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-26T18:10:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:10:53.793 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:10:53.822 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:10:53.829 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:10:53.883 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:10:53.914 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:10:53.925 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:10:53.943 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:10:53.950 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:10:53.993 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:10:54.090 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:10:54.140 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:10:54.168 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:10:54.262 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:10:54.294 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:10:54.346 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:10:54.348 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:10:54.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:10:54.393 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:10:54.442 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:10:54.545 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:10:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:10:54.578 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:10:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:10:54.685 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:10:54.686 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:10:54.714 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:10:54.734 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-04-26T18:10:54.782 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:10:54.819 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:10:54.854 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:10:54.872 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:10:54.880 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-04-26T18:10:54.881 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-04-26T18:10:54.920 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:10:54.926 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:10:54.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:10:54.962 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:10:55.005 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:10:55.038 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:10:55.053 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:10:55.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:10:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:10:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:10:55.135 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:10:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:10:55.175 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:10:55.222 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:10:55.252 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:10:55.296 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:10:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:10:55.366 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:10:55.372 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:10:55.406 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:10:55.450 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:10:55.476 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:10:55.478 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:10:55.549 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:10:55.593 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:10:55.668 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:10:55.743 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:10:55.781 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:10:55.802 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:10:55.816 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:10:55.862 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:10:55.892 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:10:55.941 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:10:55.985 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:10:56.025 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:10:56.053 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:10:56.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:10:56.184 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:10:56.214 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:10:56.249 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:10:56.329 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:10:56.360 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:10:56.375 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:10:56.404 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:10:56.432 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:10:56.465 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:10:56.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:10:56.508 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:10:56.537 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:10:56.556 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:10:56.575 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:10:56.617 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:10:56.634 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:10:56.701 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-04-26T18:10:56.701 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-04-26T18:10:56.782 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:10:56.788 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:10:56.845 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:10:56.852 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:10:56.899 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:10:56.963 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:10:56.996 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:10:57.032 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:10:57.079 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:10:57.115 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:10:57.158 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:10:57.200 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:10:57.334 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:10:57.380 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:10:57.411 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:10:57.414 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:10:57.468 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:10:57.498 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:10:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:10:57.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:10:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:10:57.595 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:10:57.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:10:57.634 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:10:57.660 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:10:57.664 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:10:57.685 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:10:57.759 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:10:57.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:10:57.867 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:10:57.895 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:10:57.923 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:10:57.925 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:10:57.955 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:10:57.987 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:10:57.997 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:10:58.026 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:10:58.035 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:10:58.062 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:10:58.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:10:58.096 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:10:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:10:58.131 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:10:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:10:58.164 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:10:58.189 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:10:58.206 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:10:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:10:58.240 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:10:58.328 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:10:58.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:10:58.374 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:10:58.411 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:10:58.441 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:10:58.483 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:10:58.502 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:10:58.549 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:10:58.582 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:10:58.616 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:10:58.647 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:10:58.656 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:10:58.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:10:58.704 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:10:58.706 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:10:58.786 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:10:58.914 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:10:58.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:10:58.976 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:10:59.010 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:10:59.102 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:10:59.156 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:10:59.185 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:10:59.221 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:10:59.253 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:10:59.296 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:10:59.330 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:10:59.419 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:10:59.420 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:10:59.467 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:10:59.469 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:10:59.499 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:10:59.500 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:10:59.548 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:10:59.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:10:59.658 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:10:59.659 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:10:59.710 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:10:59.747 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:10:59.853 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:11:00.029 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:11:00.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:11:00.146 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:11:00.476 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:11:00.523 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:11:00.556 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:11:00.667 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:11:00.738 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:11:00.780 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:11:00.829 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:11:00.867 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:11:00.907 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:11:01.051 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:11:01.072 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:11:01.144 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:11:01.832 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:11:01.958 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:11:02.116 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:11:02.197 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:11:02.225 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:11:02.258 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:11:02.587 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:11:02.688 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-04-26T18:11:02.759 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:11:02.779 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:02.797 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:11:02.851 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:11:02.869 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:02.885 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:11:02.892 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:11:02.911 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:11:02.928 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:11:02.953 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:11:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:11:03.283 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:11:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:11:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:11:03.387 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:11:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:11:03.408 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:11:03.441 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:03.508 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:11:03.512 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:03.538 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:11:03.542 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:11:03.629 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:11:03.659 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:11:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:11:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:03.809 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:11:03.865 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:11:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:03.893 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:11:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:11:03.958 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:11:03.978 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:11:04.003 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:11:04.056 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:11:04.078 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:11:04.100 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:11:04.295 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:11:04.331 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:11:04.441 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:11:04.468 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:11:04.507 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:04.530 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:11:04.561 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:11:04.576 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:04.612 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:11:04.628 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:11:04.685 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:11:04.732 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:11:04.773 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:11:04.801 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:11:04.828 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:11:04.898 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:11:04.949 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:11:04.980 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:11:05.014 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:11:05.098 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:11:05.123 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:11:05.159 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:11:05.187 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:11:05.255 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:11:05.310 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:11:05.384 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:11:05.515 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:11:05.562 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:11:05.847 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:05.880 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:06.083 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:11:06.166 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:11:06.196 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:11:06.230 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:11:06.268 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:11:06.311 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:11:06.425 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:06.456 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:11:06.468 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:06.522 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:11:06.563 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:06.563 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:11:06.563 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:11:06.563 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:06.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:11:06.594 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:06.641 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:11:06.709 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:11:06.756 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:11:06.792 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:11:06.836 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:11:06.878 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:11:06.980 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:07.010 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:07.012 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:11:07.058 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:11:07.089 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:11:07.130 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:11:07.274 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:11:07.588 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:07.647 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:07.707 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:11:07.739 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:07.739 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:11:07.739 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:11:07.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:07.764 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:11:07.790 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:07.805 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:11:07.830 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:11:07.912 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:11:08.043 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:11:08.079 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:11:08.106 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:11:08.141 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:11:08.180 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:11:08.212 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:11:08.242 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:11:08.277 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:11:08.310 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:11:08.351 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:11:08.386 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:11:08.471 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:11:08.524 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:11:08.556 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:11:08.603 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:11:08.700 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:11:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:11:09.458 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:11:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:11:09.539 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:11:09.636 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:11:09.689 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:11:09.775 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:11:09.809 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:11:09.977 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:11:09.999 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:11:10.076 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:11:10.712 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:11:10.830 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:11:12.045 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:11:12.130 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:11:12.157 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:11:12.191 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:11:12.501 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:11:12.547 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:12.636 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:11:12.667 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:11:12.703 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:11:13.180 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:11:13.208 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:13.278 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:11:13.301 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:11:15.569 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:15.606 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:16.143 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:11:16.186 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:16.276 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:11:16.276 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:11:16.276 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:11:16.276 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:16.304 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:18.015 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:18.015 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-04-26T18:11:18.015 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-04-26T18:11:18.016 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:18.182 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:18.199 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:18.199 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:18.200 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:11:18.200 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:19.702 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:19.728 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:19.729 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:19.729 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:11:19.729 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:19.729 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:19.729 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:19.818 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:19.818 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-26T18:11:19.819 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:19.961 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:19.984 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:19.984 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:19.984 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:11:19.984 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:19.984 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:19.985 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:19.994 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:20.011 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:20.011 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:20.012 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:11:20.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:20.437 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:20.459 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:20.460 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:20.460 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:11:20.460 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:20.460 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:20.460 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:20.575 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:11:20.624 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:11:20.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:11:20.866 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:11:20.913 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:11:20.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:11:21.071 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:21.201 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:21.262 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:11:21.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:11:21.323 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:11:21.547 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:21.574 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:21.574 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:21.574 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:11:21.574 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:21.575 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:21.575 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:21.767 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:21.788 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:21.815 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:21.820 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:11:21.838 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:21.838 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:21.839 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:11:21.839 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:21.839 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:21.890 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:11:21.919 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:22.438 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:11:22.497 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:11:22.559 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:11:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:11:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:11:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:11:22.959 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:22.971 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:22.996 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:23.120 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:23.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:11:23.190 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:11:23.217 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:11:23.663 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:23.691 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:23.724 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:11:23.795 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:11:23.823 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:23.968 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:23.997 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:24.098 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:24.267 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:11:24.299 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:24.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:24.366 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:24.378 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:24.475 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:11:24.531 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:11:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:25.453 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:11:25.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:25.517 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:25.518 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:25.518 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:11:25.518 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:25.518 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:25.518 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:25.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:25.584 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:25.584 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:25.584 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:11:25.584 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:25.629 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:25.651 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:25.651 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:25.652 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:11:25.652 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:25.652 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:25.652 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:25.971 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:25.994 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:26.278 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:11:26.302 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:26.326 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:26.326 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:26.326 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:11:26.326 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:26.327 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:26.327 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:26.386 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:26.397 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:11:26.560 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:11:27.543 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:11:27.572 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:27.596 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:27.596 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:27.596 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:11:27.596 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:27.596 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:27.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:27.662 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:27.662 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:11:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:27.715 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:27.738 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:27.738 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:11:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:27.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-04-26T18:11:27.742 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:27.921 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:27.937 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:11:27.937 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:27.937 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:11:27.937 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:28.057 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:28.081 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:29.391 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:11:29.416 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:29.642 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:11:29.664 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:30.110 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:11:30.146 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:30.247 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:11:30.299 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:11:30.353 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:11:30.547 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:11:30.602 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:11:30.608 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:11:30.649 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:11:30.671 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:11:30.690 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:11:30.722 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:11:30.750 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:11:30.758 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:30.897 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:30.936 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:11:30.937 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:11:30.968 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:11:30.988 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:11:30.996 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:11:31.026 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:11:31.486 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:11:31.556 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:11:31.586 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:31.649 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:31.690 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:31.711 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:32.535 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:11:32.629 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:11:32.669 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:11:32.670 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:11:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:11:32.800 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:32.892 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:32.925 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:32.925 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:32.970 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:11:33.011 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:11:33.012 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:33.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:11:33.081 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:33.083 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:11:33.111 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:11:33.313 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:11:33.357 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:11:33.386 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:11:33.626 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:11:33.654 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:33.753 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:11:33.916 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:11:33.948 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:11:33.971 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:34.024 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:34.035 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:11:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:34.132 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:11:34.181 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:11:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:11:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:11:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:11:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:11:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:35.149 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:11:35.149 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:35.150 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:11:35.150 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:35.150 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:11:35.150 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:35.195 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:35.199 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:11:35.216 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:11:35.217 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:35.217 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:11:35.217 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:11:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:35.262 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:35.283 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:11:35.284 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:35.284 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:11:35.284 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:35.284 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:11:35.284 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:35.588 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:11:35.610 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:11:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:11:39.215 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:11:39.216 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:11:39.217 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:11:39.230 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:11:39.231 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:11:39.232 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:11:39.233 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:11:39.234 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:11:39.235 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:11:39.236 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:11:39.237 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:11:39.238 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:11:39.239 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:11:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:11:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:11:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:39.240 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:11:40.192 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:11:40.241 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:11:40.282 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:11:40.315 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:11:40.342 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:11:40.522 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:11:40.572 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:11:40.602 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:11:41.182 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:11:41.183 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:11:41.184 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:11:41.185 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:11:41.197 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:11:41.198 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:11:41.199 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:11:41.200 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:11:41.201 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:11:41.202 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:11:41.203 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:11:41.204 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:11:41.205 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:41.206 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:11:41.237 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:41.275 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:41.295 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:11:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:41.414 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-04-26T18:11:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.414 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.415 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.416 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:11:41.417 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.418 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:11:41.419 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:11:41.420 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:11:41.421 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:11:41.422 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:11:41.423 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:11:41.424 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:11:41.425 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:11:41.426 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:11:41.427 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:11:41.427 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:11:41.427 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-04-26T18:11:41.795 DEBUG:teuthology.parallel:result is None 2024-04-26T18:11:42.117 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:11:42.198 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:11:42.247 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:11:42.336 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:11:42.370 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:11:42.370 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:42.454 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:42.479 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:42.571 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:11:42.635 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:43.308 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:43.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:43.308 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.309 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:11:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.312 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:11:43.313 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.314 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:11:43.315 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:43.316 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:11:43.317 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:11:43.318 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:11:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:11:43.320 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T18:11:43.722 DEBUG:teuthology.parallel:result is None 2024-04-26T18:11:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:48.603 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:11:48.604 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:11:48.605 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:11:48.606 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:11:48.607 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:11:48.608 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:11:48.609 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:11:48.610 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:11:48.611 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:11:48.612 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:11:48.613 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:11:48.614 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:11:48.615 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:11:50.703 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.704 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.705 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.706 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:11:50.707 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:11:50.708 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:11:50.709 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:11:50.710 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:11:50.711 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:11:50.712 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:11:50.713 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:11:50.714 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T18:11:50.715 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-04-26T18:11:51.105 DEBUG:teuthology.parallel:result is None 2024-04-26T18:11:51.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:51.249 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:11:51.275 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.276 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.276 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:11:51.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:51.496 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:11:51.518 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.518 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.518 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:11:51.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:51.659 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:11:51.685 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.685 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:11:51.686 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:11:51.687 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T18:11:51.688 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:11:51.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:11:51.720 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:11:51.720 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:11:51.747 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:11:51.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:11:51.779 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T18:11:51.779 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:11:51.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:11:51.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:11:51.881 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:11:51.881 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:11:51.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:11:51.974 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:11:51.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:11:52.003 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:11:52.073 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T18:11:52.074 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:11:52.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:11:52.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:11:52.174 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:11:52.174 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:11:52.199 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:11:52.265 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:11:52.265 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:11:52.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:11:52.360 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T18:11:52.360 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:11:52.360 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:11:52.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:11:52.459 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:11:52.459 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:11:52.485 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:11:52.550 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:11:52.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:11:52.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:11:52.646 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T18:11:52.769 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-26T18:11:52.770 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:52.770 INFO:tasks.cephadm:Cluster fsid is 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:11:52.770 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T18:11:52.770 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.120', 'mon.c': '172.21.15.170'} 2024-04-26T18:11:52.771 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-04-26T18:11:52.771 INFO:tasks.cephadm:First mgr is x 2024-04-26T18:11:52.771 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T18:11:52.771 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-26T18:11:52.801 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-04-26T18:11:52.824 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-04-26T18:11:52.852 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T18:11:52.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:52.990 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78913/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3267-gbc94a2a9', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3267.gbc94a2a9'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 08:22:46.917786', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer-fixes', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-26T18:11:53.218 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiescer-fixes, sha1 bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:11:53.219 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm 2024-04-26T18:11:53.219 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:54.511 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:11 /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:54.512 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:55.752 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:11 /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:55.752 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:56.879 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:11 /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:56.879 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:56.899 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:56.919 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:11:56.954 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 on all hosts... 2024-04-26T18:11:56.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:11:56.957 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:11:56.961 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:11:58.501 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:11:58.547 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:11:58.833 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:12:33.274 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-26T18:12:33.274 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:12:33.274 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:12:33.274 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-04-26T18:12:33.275 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:12:33.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-26T18:12:33.275 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-26T18:12:37.734 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-04-26T18:12:37.735 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-26T18:12:45.653 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:12:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:12:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:12:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-26T18:12:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:12:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:12:45.654 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:12:45.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-26T18:12:45.717 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-04-26T18:12:45.750 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-04-26T18:12:45.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-26T18:12:45.806 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-04-26T18:12:45.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-04-26T18:12:45.861 INFO:tasks.cephadm:Writing seed config... 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T18:12:45.862 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T18:12:45.863 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T18:12:45.864 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T18:12:45.865 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T18:12:45.866 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T18:12:45.866 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T18:12:45.866 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:12:45.866 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T18:12:45.882 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 755ef512-03f8-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T18:12:45.882 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T18:12:45.924 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T18:12:45.966 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T18:12:45.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 -v bootstrap --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:46.162 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:12:46.163 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', '-v', 'bootstrap', '--fsid', '755ef512-03f8-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-04-26T18:12:46.163 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T18:12:46.163 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-04-26T18:12:46.189 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:12:46.189 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-04-26T18:12:46.189 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-04-26T18:12:46.197 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:12:46.197 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:12:46.203 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:12:46.203 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:12:46.210 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-26T18:12:46.216 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-26T18:12:46.217 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:12:46.217 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-04-26T18:12:46.240 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:12:46.240 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T18:12:46.240 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-04-26T18:12:46.240 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-04-26T18:12:46.247 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:12:46.247 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:12:46.253 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:12:46.253 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:12:46.260 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-26T18:12:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 139873058765984 on /run/cephadm/755ef512-03f8-11ef-bc93-c7b262605968.lock 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 139873058765984 acquired on /run/cephadm/755ef512-03f8-11ef-bc93-c7b262605968.lock 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-04-26T18:12:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-26T18:12:46.270 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-04-26T18:12:46.271 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-04-26T18:12:46.274 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T18:12:46.274 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T18:12:46.274 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T18:12:46.276 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:12:46.277 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-26T18:12:46.277 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-26T18:12:46.277 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T18:12:46.277 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T18:12:46.278 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:12:46.764 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:12:46.764 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:12:46.764 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T18:12:46.764 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T18:12:46.764 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:449452a284afe5840255d087488a224329a4679c5f5d200bb867a639c75ddfd2 2024-04-26T18:12:46.765 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:12:46.765 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T18:12:47.726 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:12:47.726 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:12:47.726 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T18:12:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-04-26T18:12:48.727 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCh7itmPAi/ABAAgCURC9yyk0/G4z6lPJikGA== 2024-04-26T18:12:50.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCh7itmzCpQMxAAf4Zt5TaUWpK86Ufj87OJww== 2024-04-26T18:12:51.304 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCi7itmzVC0LhAAczKdH3m2z5RX0VAbnEnrAw== 2024-04-26T18:12:51.306 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:12:52.219 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:12:52.220 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-04-26T18:12:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-04-26T18:12:53.475 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T18:12:53.639 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T18:12:53.818 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target. 2024-04-26T18:12:53.818 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target. 2024-04-26T18:12:54.077 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a 2024-04-26T18:12:54.077 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T18:12:54.270 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target.wants/ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968@.service. 2024-04-26T18:12:54.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 systemd[1]: Starting Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:12:54.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 podman[28777]: 2024-04-26 18:12:54.519906435 +0000 UTC m=+0.112607081 container create 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:12:54.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 podman[28777]: 2024-04-26 18:12:54.426918341 +0000 UTC m=+0.019619009 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:12:54.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 podman[28777]: 2024-04-26 18:12:54.813768778 +0000 UTC m=+0.406469424 container init 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T18:12:54.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 podman[28777]: 2024-04-26 18:12:54.816926169 +0000 UTC m=+0.409626816 container start 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T18:12:54.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 ceph-mon[28795]: mkfs 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:54.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 ceph-mon[28795]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:12:54.926 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:12:54.926 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T18:12:54.932 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:12:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:12:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T18:12:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:12:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-04-26T18:12:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-04-26T18:12:55.166 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 bash[28777]: 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb 2024-04-26T18:12:55.166 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:54 smithi037 systemd[1]: Started Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-04-26T18:12:55.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.597219s) 2024-04-26T18:12:55.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T18:12:55.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.384+0000 7f803c7d8700 1 Processor -- start 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.384+0000 7f803c7d8700 1 -- start start 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.384+0000 7f803c7d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.384+0000 7f803c7d8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8034107c50 con 0x7f8034107260 2024-04-26T18:12:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f803a574700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f803a574700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37610/0 (socket says 172.21.15.37:37610) 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f803a574700 1 -- 172.21.15.37:0/863102375 learned_addr learned my addr 172.21.15.37:0/863102375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f803a574700 1 -- 172.21.15.37:0/863102375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803410a1f0 con 0x7f8034107260 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f803a574700 1 --2- 172.21.15.37:0/863102375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f80300098a0 tx=0x7f8030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7f7e13f8616efd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f8039572700 1 -- 172.21.15.37:0/863102375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030004160 con 0x7f8034107260 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f8039572700 1 -- 172.21.15.37:0/863102375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030004300 con 0x7f8034107260 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.385+0000 7f8039572700 1 -- 172.21.15.37:0/863102375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80300036a0 con 0x7f8034107260 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 -- 172.21.15.37:0/863102375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 msgr2=0x7f8034107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 --2- 172.21.15.37:0/863102375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f80300098a0 tx=0x7f8030009bb0 comp rx=0 tx=0).stop 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 -- 172.21.15.37:0/863102375 shutdown_connections 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 --2- 172.21.15.37:0/863102375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f8034107680 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 -- 172.21.15.37:0/863102375 >> 172.21.15.37:0/863102375 conn(0x7f8034102b90 msgr2=0x7f8034104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:55.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 -- 172.21.15.37:0/863102375 shutdown_connections 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 -- 172.21.15.37:0/863102375 wait complete. 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.386+0000 7f803c7d8700 1 Processor -- start 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803c7d8700 1 -- start start 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803c7d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803c7d8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f803410a720 con 0x7f8034107260 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803a574700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803a574700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37614/0 (socket says 172.21.15.37:37614) 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803a574700 1 -- 172.21.15.37:0/2992664643 learned_addr learned my addr 172.21.15.37:0/2992664643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803a574700 1 -- 172.21.15.37:0/2992664643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803419cc50 con 0x7f8034107260 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803a574700 1 --2- 172.21.15.37:0/2992664643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f8030004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030003db0 con 0x7f8034107260 2024-04-26T18:12:55.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030011630 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803419cee0 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.387+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8030011950 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.388+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803419fbd0 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.388+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f803001a460 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.388+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f803001b780 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.388+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8034064a80 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.391+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8030031020 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.464+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f803402e790 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.465+0000 7f802b7fe700 1 -- 172.21.15.37:0/2992664643 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f803001ba20 con 0x7f8034107260 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 msgr2=0x7f803419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 --2- 172.21.15.37:0/2992664643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f8030004750 comp rx=0 tx=0).stop 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 shutdown_connections 2024-04-26T18:12:55.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 --2- 172.21.15.37:0/2992664643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8034107260 0x7f803419c710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 >> 172.21.15.37:0/2992664643 conn(0x7f8034102b90 msgr2=0x7f8034193350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 shutdown_connections 2024-04-26T18:12:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:55.466+0000 7f803c7d8700 1 -- 172.21.15.37:0/2992664643 wait complete. 2024-04-26T18:12:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-04-26T18:12:55.974 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T18:12:56.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:12:56.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: monmap epoch 1 2024-04-26T18:12:56.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:56.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: last_changed 2024-04-26T18:12:51.672210+0000 2024-04-26T18:12:56.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: min_mon_release 19 (squid) 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: election_strategy: 1 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: fsmap 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: mgrmap e1: no daemons active 2024-04-26T18:12:56.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:55 smithi037 ceph-mon[28795]: from='client.? 172.21.15.37:0/2992664643' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T18:12:56.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:56 smithi037 ceph-mon[28795]: from='client.? 172.21.15.37:0/3861423196' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T18:12:56.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:56 smithi037 ceph-mon[28795]: from='client.? 172.21.15.37:0/3861423196' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:12:56.990 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.369+0000 7fd31841b700 1 Processor -- start 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.369+0000 7fd31841b700 1 -- start start 2024-04-26T18:12:56.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd31841b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd31841b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd31007d8c0 con 0x7fd31007ee90 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3161b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3161b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37624/0 (socket says 172.21.15.37:37624) 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3161b7700 1 -- 172.21.15.37:0/2998212818 learned_addr learned my addr 172.21.15.37:0/2998212818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3161b7700 1 -- 172.21.15.37:0/2998212818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31007da40 con 0x7fd31007ee90 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3161b7700 1 --2- 172.21.15.37:0/2998212818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098a0 tx=0x7fd30c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5547999a4a77a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.370+0000 7fd3151b5700 1 -- 172.21.15.37:0/2998212818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c004160 con 0x7fd31007ee90 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd3151b5700 1 -- 172.21.15.37:0/2998212818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c004300 con 0x7fd31007ee90 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd3151b5700 1 -- 172.21.15.37:0/2998212818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c0036a0 con 0x7fd31007ee90 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 -- 172.21.15.37:0/2998212818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 msgr2=0x7fd31007d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 --2- 172.21.15.37:0/2998212818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098a0 tx=0x7fd30c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:12:56.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 -- 172.21.15.37:0/2998212818 shutdown_connections 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 --2- 172.21.15.37:0/2998212818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31007d2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 -- 172.21.15.37:0/2998212818 >> 172.21.15.37:0/2998212818 conn(0x7fd3101044f0 msgr2=0x7fd310106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 -- 172.21.15.37:0/2998212818 shutdown_connections 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.371+0000 7fd31841b700 1 -- 172.21.15.37:0/2998212818 wait complete. 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd31841b700 1 Processor -- start 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd31841b700 1 -- start start 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd31841b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd31841b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd31007df70 con 0x7fd31007ee90 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd3161b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd3161b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37636/0 (socket says 172.21.15.37:37636) 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd3161b7700 1 -- 172.21.15.37:0/3861423196 learned_addr learned my addr 172.21.15.37:0/3861423196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd3161b7700 1 -- 172.21.15.37:0/3861423196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31019ee00 con 0x7fd31007ee90 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.372+0000 7fd3161b7700 1 --2- 172.21.15.37:0/3861423196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd31007e360 tx=0x7fd30c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:56.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c003db0 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c011630 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31019f090 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c011930 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a1d80 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd30c011b10 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.373+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd30c01b880 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.374+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd310064a80 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.376+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd30c011e00 con 0x7fd31007ee90 2024-04-26T18:12:56.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.447+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd31002e720 con 0x7fd31007ee90 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.450+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fd30c004260 con 0x7fd31007ee90 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd3077fe700 1 -- 172.21.15.37:0/3861423196 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd30c037580 con 0x7fd31007ee90 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 msgr2=0x7fd31019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd31841b700 1 --2- 172.21.15.37:0/3861423196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd31007e360 tx=0x7fd30c004750 comp rx=0 tx=0).stop 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 shutdown_connections 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd31841b700 1 --2- 172.21.15.37:0/3861423196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd31007ee90 0x7fd31019e8c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.451+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 >> 172.21.15.37:0/3861423196 conn(0x7fd3101044f0 msgr2=0x7fd310106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.452+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 shutdown_connections 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:56.452+0000 7fd31841b700 1 -- 172.21.15.37:0/3861423196 wait complete. 2024-04-26T18:12:56.998 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-04-26T18:12:58.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.361+0000 7f7d1ddbf700 1 Processor -- start 2024-04-26T18:12:58.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.361+0000 7f7d1ddbf700 1 -- start start 2024-04-26T18:12:58.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d1ddbf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:58.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d1ddbf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1807d8e0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37638/0 (socket says 172.21.15.37:37638) 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d177fe700 1 -- 172.21.15.37:0/2152408212 learned_addr learned my addr 172.21.15.37:0/2152408212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d177fe700 1 -- 172.21.15.37:0/2152408212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1807da60 con 0x7f7d1807eeb0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d177fe700 1 --2- 172.21.15.37:0/2152408212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d080098a0 tx=0x7f7d08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9bc5c4193c8371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d167fc700 1 -- 172.21.15.37:0/2152408212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08004160 con 0x7f7d1807eeb0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d167fc700 1 -- 172.21.15.37:0/2152408212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d08004300 con 0x7f7d1807eeb0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.362+0000 7f7d167fc700 1 -- 172.21.15.37:0/2152408212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08011af0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/2152408212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 msgr2=0x7f7d1807d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 --2- 172.21.15.37:0/2152408212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7d080098a0 tx=0x7f7d08009bb0 comp rx=0 tx=0).stop 2024-04-26T18:12:58.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/2152408212 shutdown_connections 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 --2- 172.21.15.37:0/2152408212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1807d310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/2152408212 >> 172.21.15.37:0/2152408212 conn(0x7f7d181045a0 msgr2=0x7f7d181069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/2152408212 shutdown_connections 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.363+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/2152408212 wait complete. 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d1ddbf700 1 Processor -- start 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d1ddbf700 1 -- start start 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d1ddbf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d1ddbf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1810f570 con 0x7f7d1807eeb0 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37640/0 (socket says 172.21.15.37:37640) 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d177fe700 1 -- 172.21.15.37:0/503065510 learned_addr learned my addr 172.21.15.37:0/503065510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:12:58.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.364+0000 7f7d177fe700 1 -- 172.21.15.37:0/503065510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1819f670 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d177fe700 1 --2- 172.21.15.37:0/503065510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d080117b0 tx=0x7f7d08011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08003af0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1819f900 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d08020070 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d08019630 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181a25f0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d08019c80 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.365+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d0800bca0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.366+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d18064a80 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.368+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7d0801a9c0 con 0x7f7d1807eeb0 2024-04-26T18:12:58.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.440+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7d181a2b90 con 0x7f7d1807eeb0 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.440+0000 7f7d14ff9700 1 -- 172.21.15.37:0/503065510 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f7d0801a430 con 0x7f7d1807eeb0 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 msgr2=0x7f7d1819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 --2- 172.21.15.37:0/503065510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d080117b0 tx=0x7f7d08011b70 comp rx=0 tx=0).stop 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 shutdown_connections 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 --2- 172.21.15.37:0/503065510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d1807eeb0 0x7f7d1819f130 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 >> 172.21.15.37:0/503065510 conn(0x7f7d181045a0 msgr2=0x7f7d18105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 shutdown_connections 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:12:57.441+0000 7f7d1ddbf700 1 -- 172.21.15.37:0/503065510 wait complete. 2024-04-26T18:12:58.124 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-04-26T18:12:58.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:57 smithi037 ceph-mon[28795]: from='client.? 172.21.15.37:0/503065510' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:58.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 systemd[1]: Stopping Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:12:58.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a[28791]: 2024-04-26T18:12:58.431+0000 7f8308dc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:12:58.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a[28791]: 2024-04-26T18:12:58.431+0000 7f8308dc0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 podman[29044]: 2024-04-26 18:12:58.647408454 +0000 UTC m=+0.361760611 container died 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:12:59.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 podman[29044]: 2024-04-26 18:12:58.989915874 +0000 UTC m=+0.704268068 container cleanup 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:12:59.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:58 smithi037 bash[29044]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a 2024-04-26T18:12:59.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 podman[29058]: 2024-04-26 18:12:59.157374293 +0000 UTC m=+0.505725882 container remove 5cc56c8bbc96ac0e841853643b52a236391c8571b88acc4226803b4897b3f1bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T18:12:59.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T18:12:59.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 systemd[1]: Stopped Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:12:59.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 systemd[1]: Starting Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:12:59.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 podman[29152]: 2024-04-26 18:12:59.616733841 +0000 UTC m=+0.102525647 container create 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T18:12:59.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 podman[29152]: 2024-04-26 18:12:59.532386547 +0000 UTC m=+0.018178376 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:12:59.914 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T18:13:00.129 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 podman[29152]: 2024-04-26 18:12:59.81896827 +0000 UTC m=+0.304760080 container init 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 podman[29152]: 2024-04-26 18:12:59.822015199 +0000 UTC m=+0.307807007 container start 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: pidfile_write: ignore empty --pid-file 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: load: jerasure load: lrc 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Git sha 0 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: DB SUMMARY 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: DB Session ID: WU6GG423U0HVTESEIZM0 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T18:13:00.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106028 ; 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.env: 0x56470c6a5cc0 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.info_log: 0x56470eaa5d80 2024-04-26T18:13:00.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.statistics: (nil) 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.use_fsync: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:13:00.133 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.db_log_dir: 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.wal_dir: 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.write_buffer_manager: 0x56470ea770e0 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:13:00.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.unordered_write: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.row_cache: None 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.wal_filter: None 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:13:00.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.wal_compression: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:13:00.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_open_files: -1 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:13:00.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Compression algorithms supported: 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kZSTD supported: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:13:00.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.merge_operator: 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_filter: None 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56470eaa5f40) 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:13:00.139 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x56470eaa91f0 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:13:00.140 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-04-26T18:13:00.141 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression: NoCompression 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.num_levels: 7 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:13:00.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:13:00.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:13:00.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:13:00.145 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:13:00.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.ttl: 2592000 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:13:00.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T18:13:00.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 47a68f7d-53d9-4d78-b6b2-0ab982a8e251 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155179854000, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155179855099, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98969, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91148, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714155179, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "47a68f7d-53d9-4d78-b6b2-0ab982a8e251", "db_session_id": "WU6GG423U0HVTESEIZM0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155179855183, "job": 1, "event": "recovery_finished"} 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56470ebc6000 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: DB pointer 0x56470ebb2000 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:13:00.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:13:00.150 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 100.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 100.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.151 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 30.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 30.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x56470eaa91f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.152 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???) e1 preinit fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).mds e1 new map 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).mds e1 print_map 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:13:00.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 bash[29152]: 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 systemd[1]: Started Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: monmap epoch 1 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: last_changed 2024-04-26T18:12:51.672210+0000 2024-04-26T18:13:00.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: min_mon_release 19 (squid) 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: election_strategy: 1 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: fsmap 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T18:13:00.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:12:59 smithi037 ceph-mon[29171]: mgrmap e1: no daemons active 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff714c44700 1 Processor -- start 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff714c44700 1 -- start start 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff714c44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff714c44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff710109960 con 0x7ff710108f70 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff70e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff70e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37642/0 (socket says 172.21.15.37:37642) 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.354+0000 7ff70e59c700 1 -- 172.21.15.37:0/3909432410 learned_addr learned my addr 172.21.15.37:0/3909432410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:00.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.355+0000 7ff70e59c700 1 -- 172.21.15.37:0/3909432410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71010a170 con 0x7ff710108f70 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.355+0000 7ff70e59c700 1 --2- 172.21.15.37:0/3909432410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7000098a0 tx=0x7ff700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f6f40f8c4a4b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.355+0000 7ff70d59a700 1 -- 172.21.15.37:0/3909432410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff700004160 con 0x7ff710108f70 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.355+0000 7ff70d59a700 1 -- 172.21.15.37:0/3909432410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff700004300 con 0x7ff710108f70 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.355+0000 7ff70d59a700 1 -- 172.21.15.37:0/3909432410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff700011af0 con 0x7ff710108f70 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- 172.21.15.37:0/3909432410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 msgr2=0x7ff710109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 --2- 172.21.15.37:0/3909432410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7000098a0 tx=0x7ff700009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- 172.21.15.37:0/3909432410 shutdown_connections 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 --2- 172.21.15.37:0/3909432410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff710109390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- 172.21.15.37:0/3909432410 >> 172.21.15.37:0/3909432410 conn(0x7ff7101044f0 msgr2=0x7ff710106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- 172.21.15.37:0/3909432410 shutdown_connections 2024-04-26T18:13:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- 172.21.15.37:0/3909432410 wait complete. 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 Processor -- start 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.356+0000 7ff714c44700 1 -- start start 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff714c44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff714c44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff71010d520 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff70e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff70e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37658/0 (socket says 172.21.15.37:37658) 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff70e59c700 1 -- 172.21.15.37:0/1434669590 learned_addr learned my addr 172.21.15.37:0/1434669590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff70e59c700 1 -- 172.21.15.37:0/1434669590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71019d400 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff70e59c700 1 --2- 172.21.15.37:0/1434669590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff700003f40 tx=0x7ff700011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7000038b0 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff700003a50 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff70001a5c0 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.357+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71019d690 con 0x7ff710108f70 2024-04-26T18:13:00.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.358+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7101a0380 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.358+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff70001ac00 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.358+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff700019700 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.358+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff710064a80 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.361+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff700019900 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.431+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff7101a0550 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.433+0000 7ff6ff7fe700 1 -- 172.21.15.37:0/1434669590 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff700019ae0 con 0x7ff710108f70 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 msgr2=0x7ff71019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 --2- 172.21.15.37:0/1434669590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff700003f40 tx=0x7ff700011890 comp rx=0 tx=0).stop 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 shutdown_connections 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 --2- 172.21.15.37:0/1434669590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff710108f70 0x7ff71019cec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 >> 172.21.15.37:0/1434669590 conn(0x7ff7101044f0 msgr2=0x7ff710105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 shutdown_connections 2024-04-26T18:13:00.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:00.434+0000 7ff714c44700 1 -- 172.21.15.37:0/1434669590 wait complete. 2024-04-26T18:13:00.979 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T18:13:00.998 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:13:00.999 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-04-26T18:13:00.999 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T18:13:00.999 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T18:13:01.259 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x 2024-04-26T18:13:01.259 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T18:13:01.416 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968.target.wants/ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-755ef512-03f8-11ef-bc93-c7b262605968@.service. 2024-04-26T18:13:01.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:01 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1434669590' entity='client.admin' 2024-04-26T18:13:01.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:01 smithi037 podman[29387]: 2024-04-26 18:13:01.558634035 +0000 UTC m=+0.017939284 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:13:02.000 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:13:02.000 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T18:13:02.007 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:13:02.007 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:13:02.007 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T18:13:02.007 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:13:02.016 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:13:02.016 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T18:13:02.024 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:13:02.024 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:13:02.024 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T18:13:02.024 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T18:13:02.024 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-04-26T18:13:02.025 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-04-26T18:13:02.065 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:01 smithi037 podman[29387]: 2024-04-26 18:13:01.904139727 +0000 UTC m=+0.363444960 container init 8b820eaa54519bc7e0407ecb8168a9f910f44f058bbb26df680ab839d72a2be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:13:02.065 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:01 smithi037 podman[29387]: 2024-04-26 18:13:01.906724994 +0000 UTC m=+0.366030224 container start 8b820eaa54519bc7e0407ecb8168a9f910f44f058bbb26df680ab839d72a2be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T18:13:02.066 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:01 smithi037 bash[29387]: 8b820eaa54519bc7e0407ecb8168a9f910f44f058bbb26df680ab839d72a2be4 2024-04-26T18:13:02.066 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:01 smithi037 systemd[1]: Started Ceph mgr.x for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:13:02.368 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:02.064+0000 7f72f5738200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:13:02.368 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:02.259+0000 7f72f5738200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:02.368 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:02.367+0000 7f72f5738200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:13:02.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1980365350' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:13:02.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:02.698+0000 7f72f5738200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:13:02.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:02.790+0000 7f72f5738200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:13:03.090 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:13:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:13:03.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:13:03.102 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.103 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:12:54.870380+0000", 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:03.104 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff4558d0700 1 Processor -- start 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff4558d0700 1 -- start start 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff4558d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff4558d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff450107c20 con 0x7ff450107230 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff44effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff44effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37682/0 (socket says 172.21.15.37:37682) 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.404+0000 7ff44effd700 1 -- 172.21.15.37:0/2119450825 learned_addr learned my addr 172.21.15.37:0/2119450825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.405+0000 7ff44effd700 1 -- 172.21.15.37:0/2119450825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff450107da0 con 0x7ff450107230 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.405+0000 7ff44effd700 1 --2- 172.21.15.37:0/2119450825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff44000f960 tx=0x7ff44000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b107718ffcedea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.405+0000 7ff44dffb700 1 -- 172.21.15.37:0/2119450825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44001c070 con 0x7ff450107230 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.405+0000 7ff44dffb700 1 -- 172.21.15.37:0/2119450825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff440004030 con 0x7ff450107230 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.405+0000 7ff44dffb700 1 -- 172.21.15.37:0/2119450825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff440003c80 con 0x7ff450107230 2024-04-26T18:13:03.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 -- 172.21.15.37:0/2119450825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 msgr2=0x7ff450107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 --2- 172.21.15.37:0/2119450825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff44000f960 tx=0x7ff44000fc70 comp rx=0 tx=0).stop 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 -- 172.21.15.37:0/2119450825 shutdown_connections 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 --2- 172.21.15.37:0/2119450825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff450107650 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 -- 172.21.15.37:0/2119450825 >> 172.21.15.37:0/2119450825 conn(0x7ff450102b30 msgr2=0x7ff450104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 -- 172.21.15.37:0/2119450825 shutdown_connections 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 -- 172.21.15.37:0/2119450825 wait complete. 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.406+0000 7ff4558d0700 1 Processor -- start 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff4558d0700 1 -- start start 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff4558d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff4558d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010b090 con 0x7ff450107230 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff44effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff44effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37692/0 (socket says 172.21.15.37:37692) 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff44effd700 1 -- 172.21.15.37:0/1980365350 learned_addr learned my addr 172.21.15.37:0/1980365350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:03.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff44effd700 1 -- 172.21.15.37:0/1980365350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45019b1e0 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff44effd700 1 --2- 172.21.15.37:0/1980365350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff440006fa0 tx=0x7ff4400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.407+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44001c040 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45019b470 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff440025070 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4400205c0 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45019e160 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff440020840 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.408+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff440014070 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.409+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff450064a80 con 0x7ff450107230 2024-04-26T18:13:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.411+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff44002acf0 con 0x7ff450107230 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.487+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff45002e790 con 0x7ff450107230 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.487+0000 7ff4548ce700 1 -- 172.21.15.37:0/1980365350 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff4400179e0 con 0x7ff450107230 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 msgr2=0x7ff45019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 --2- 172.21.15.37:0/1980365350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff440006fa0 tx=0x7ff4400044c0 comp rx=0 tx=0).stop 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 shutdown_connections 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 --2- 172.21.15.37:0/1980365350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff450107230 0x7ff45019aca0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 >> 172.21.15.37:0/1980365350 conn(0x7ff450102b30 msgr2=0x7ff450103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 shutdown_connections 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:02.489+0000 7ff4558d0700 1 -- 172.21.15.37:0/1980365350 wait complete. 2024-04-26T18:13:03.108 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-04-26T18:13:04.340 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:04.048+0000 7f72f5738200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:13:04.341 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:04.223+0000 7f72f5738200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:04.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:04.400+0000 7f72f5738200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:13:04.697 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:04.697+0000 7f72f5738200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:13:04.949 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:04.792+0000 7f72f5738200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:13:05.220 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:05.005+0000 7f72f5738200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:13:05.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:05.219+0000 7f72f5738200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:05.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:05 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1126069308' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:13:06.196 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:05.858+0000 7f72f5738200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:13:06.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:06.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:13:06.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:13:06.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:12:54.870380+0000", 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:06.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a90d4700 1 Processor -- start 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a90d4700 1 -- start start 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a90d4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a90d4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4076d80 con 0x7f62a4109a20 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36320/0 (socket says 172.21.15.37:36320) 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.518+0000 7f62a2d9d700 1 -- 172.21.15.37:0/4248282118 learned_addr learned my addr 172.21.15.37:0/4248282118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.519+0000 7f62a2d9d700 1 -- 172.21.15.37:0/4248282118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a410c350 con 0x7f62a4109a20 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.519+0000 7f62a2d9d700 1 --2- 172.21.15.37:0/4248282118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d060aee0b572d4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.519+0000 7f62a1d9b700 1 -- 172.21.15.37:0/4248282118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294004160 con 0x7f62a4109a20 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.519+0000 7f62a1d9b700 1 -- 172.21.15.37:0/4248282118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6294004300 con 0x7f62a4109a20 2024-04-26T18:13:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.519+0000 7f62a1d9b700 1 -- 172.21.15.37:0/4248282118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294011af0 con 0x7f62a4109a20 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 -- 172.21.15.37:0/4248282118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 msgr2=0x7f62a410be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 --2- 172.21.15.37:0/4248282118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 -- 172.21.15.37:0/4248282118 shutdown_connections 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 --2- 172.21.15.37:0/4248282118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a410be10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 -- 172.21.15.37:0/4248282118 >> 172.21.15.37:0/4248282118 conn(0x7f62a41034f0 msgr2=0x7f62a4105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 -- 172.21.15.37:0/4248282118 shutdown_connections 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 -- 172.21.15.37:0/4248282118 wait complete. 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.520+0000 7f62a90d4700 1 Processor -- start 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a90d4700 1 -- start start 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a90d4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a90d4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410d320 con 0x7f62a4109a20 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36336/0 (socket says 172.21.15.37:36336) 2024-04-26T18:13:06.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a2d9d700 1 -- 172.21.15.37:0/1126069308 learned_addr learned my addr 172.21.15.37:0/1126069308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a2d9d700 1 -- 172.21.15.37:0/1126069308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a419d420 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.521+0000 7f62a2d9d700 1 --2- 172.21.15.37:0/1126069308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6294011ac0 tx=0x7f6294011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.522+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294003970 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.522+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6294003b10 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.522+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a419d6b0 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.522+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294024610 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.522+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41a03a0 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.523+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6294024840 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.523+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a419db90 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.523+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f629402dd20 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.525+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6294043e50 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.600+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f62a41a0570 con 0x7f62a4109a20 2024-04-26T18:13:06.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.600+0000 7f628bfff700 1 -- 172.21.15.37:0/1126069308 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6294032030 con 0x7f62a4109a20 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.601+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 msgr2=0x7f62a419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.601+0000 7f62a90d4700 1 --2- 172.21.15.37:0/1126069308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6294011ac0 tx=0x7f6294011ba0 comp rx=0 tx=0).stop 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.601+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 shutdown_connections 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.601+0000 7f62a90d4700 1 --2- 172.21.15.37:0/1126069308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62a4109a20 0x7f62a419cee0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.601+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 >> 172.21.15.37:0/1126069308 conn(0x7f62a41034f0 msgr2=0x7f62a4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.602+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 shutdown_connections 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:05.602+0000 7f62a90d4700 1 -- 172.21.15.37:0/1126069308 wait complete. 2024-04-26T18:13:06.217 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-04-26T18:13:06.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:06.299+0000 7f72f5738200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:13:06.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:06.398+0000 7f72f5738200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:13:06.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:06.606+0000 7f72f5738200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:13:07.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.149+0000 7f72f5738200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:13:07.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.244+0000 7f72f5738200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:13:07.849 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.495+0000 7f72f5738200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:13:07.849 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.632+0000 7f72f5738200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:13:07.849 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.730+0000 7f72f5738200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:13:07.849 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:07.849+0000 7f72f5738200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:08.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:08.159+0000 7f72f5738200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:13:08.807 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:08 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3183751246' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:13:09.232 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:08.806+0000 7f72f5738200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:13:09.232 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:09.029+0000 7f72f5738200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:13:09.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:13:09.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:13:09.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:13:09.377 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:09.378 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:12:54.870380+0000", 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.590+0000 7fc99889c700 1 Processor -- start 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.590+0000 7fc99889c700 1 -- start start 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.590+0000 7fc99889c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.590+0000 7fc99889c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc994109ed0 con 0x7fc9941094e0 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36338/0 (socket says 172.21.15.37:36338) 2024-04-26T18:13:09.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99259c700 1 -- 172.21.15.37:0/2961588456 learned_addr learned my addr 172.21.15.37:0/2961588456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99259c700 1 -- 172.21.15.37:0/2961588456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99410a050 con 0x7fc9941094e0 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99259c700 1 --2- 172.21.15.37:0/2961588456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098a0 tx=0x7fc988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6d9535a9082580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99159a700 1 -- 172.21.15.37:0/2961588456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988004160 con 0x7fc9941094e0 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99159a700 1 -- 172.21.15.37:0/2961588456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc988004300 con 0x7fc9941094e0 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.591+0000 7fc99159a700 1 -- 172.21.15.37:0/2961588456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988011af0 con 0x7fc9941094e0 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 -- 172.21.15.37:0/2961588456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 msgr2=0x7fc994109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 --2- 172.21.15.37:0/2961588456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098a0 tx=0x7fc988009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 -- 172.21.15.37:0/2961588456 shutdown_connections 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 --2- 172.21.15.37:0/2961588456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc994109900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:09.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 -- 172.21.15.37:0/2961588456 >> 172.21.15.37:0/2961588456 conn(0x7fc994102c70 msgr2=0x7fc994105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 -- 172.21.15.37:0/2961588456 shutdown_connections 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.592+0000 7fc99889c700 1 -- 172.21.15.37:0/2961588456 wait complete. 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99889c700 1 Processor -- start 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99889c700 1 -- start start 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99889c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99889c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc994107d80 con 0x7fc9941094e0 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36350/0 (socket says 172.21.15.37:36350) 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99259c700 1 -- 172.21.15.37:0/3183751246 learned_addr learned my addr 172.21.15.37:0/3183751246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99259c700 1 -- 172.21.15.37:0/3183751246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99419f5e0 con 0x7fc9941094e0 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.593+0000 7fc99259c700 1 --2- 172.21.15.37:0/3183751246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117b0 tx=0x7fc988011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:09.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988003af0 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc988020070 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99419f8d0 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988019630 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a2560 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc988019c80 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.594+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc98800b720 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.595+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994064a80 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.599+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc9880042a0 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.674+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc994198600 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc9837fe700 1 -- 172.21.15.37:0/3183751246 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc98801a9c0 con 0x7fc9941094e0 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 msgr2=0x7fc99419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:09.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc99889c700 1 --2- 172.21.15.37:0/3183751246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc9880117b0 tx=0x7fc988011b70 comp rx=0 tx=0).stop 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 shutdown_connections 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc99889c700 1 --2- 172.21.15.37:0/3183751246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9941094e0 0x7fc99419f0a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.675+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 >> 172.21.15.37:0/3183751246 conn(0x7fc994102c70 msgr2=0x7fc9941956c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.676+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 shutdown_connections 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:08.676+0000 7fc99889c700 1 -- 172.21.15.37:0/3183751246 wait complete. 2024-04-26T18:13:09.383 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-04-26T18:13:09.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: Activating manager daemon x 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: mgrmap e2: x(active, starting, since 0.00310838s) 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: Manager daemon x is now available 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' 2024-04-26T18:13:09.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' 2024-04-26T18:13:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:09 smithi037 ceph-mon[29171]: from='mgr.14100 172.21.15.37:0/4008024910' entity='mgr.x' 2024-04-26T18:13:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:11 smithi037 ceph-mon[29171]: mgrmap e3: x(active, since 1.00758s) 2024-04-26T18:13:12.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:12 smithi037 ceph-mon[29171]: mgrmap e4: x(active, since 2s) 2024-04-26T18:13:15.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:15 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/830466920' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:13:20.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:20.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:20.801 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-26T18:13:20.802 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:13:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:13:20.804 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:13:20.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:12:54.870380+0000", 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.152+0000 7f3568f48700 1 Processor -- start 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.152+0000 7f3568f48700 1 -- start start 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.152+0000 7f3568f48700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.152+0000 7f3568f48700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3564076d20 con 0x7f3564109a80 2024-04-26T18:13:20.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52930/0 (socket says 172.21.15.37:52930) 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356259c700 1 -- 172.21.15.37:0/350880927 learned_addr learned my addr 172.21.15.37:0/350880927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356259c700 1 -- 172.21.15.37:0/350880927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356410c3b0 con 0x7f3564109a80 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356259c700 1 --2- 172.21.15.37:0/350880927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35540098a0 tx=0x7f3554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eba6860854c92af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356159a700 1 -- 172.21.15.37:0/350880927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3554004160 con 0x7f3564109a80 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356159a700 1 -- 172.21.15.37:0/350880927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3554004300 con 0x7f3564109a80 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.153+0000 7f356159a700 1 -- 172.21.15.37:0/350880927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3554011af0 con 0x7f3564109a80 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 -- 172.21.15.37:0/350880927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 msgr2=0x7f356410be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 --2- 172.21.15.37:0/350880927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f35540098a0 tx=0x7f3554009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 -- 172.21.15.37:0/350880927 shutdown_connections 2024-04-26T18:13:20.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 --2- 172.21.15.37:0/350880927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f356410be70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 -- 172.21.15.37:0/350880927 >> 172.21.15.37:0/350880927 conn(0x7f35641034f0 msgr2=0x7f3564105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 -- 172.21.15.37:0/350880927 shutdown_connections 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.154+0000 7f3568f48700 1 -- 172.21.15.37:0/350880927 wait complete. 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f3568f48700 1 Processor -- start 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f3568f48700 1 -- start start 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f3568f48700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f3568f48700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f356410d320 con 0x7f3564109a80 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f356259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f356259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52936/0 (socket says 172.21.15.37:52936) 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f356259c700 1 -- 172.21.15.37:0/830466920 learned_addr learned my addr 172.21.15.37:0/830466920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.155+0000 7f356259c700 1 -- 172.21.15.37:0/830466920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3564109840 con 0x7f3564109a80 2024-04-26T18:13:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f356259c700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3554009fb0 tx=0x7f3554011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35540036a0 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f355401a430 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3564107a30 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3554019550 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.156+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3564107f70 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f3554019800 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f35537fe700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 0x7f354c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f355404d890 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3564064a80 con 0x7f3564109a80 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f3561d9b700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 0x7f354c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.157+0000 7f3561d9b700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 0x7f354c03ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3558006fd0 tx=0x7f3558006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.163+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f355404d2b0 con 0x7f3564109a80 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.458+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f35641081f0 con 0x7f3564109a80 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.459+0000 7f35537fe700 1 -- 172.21.15.37:0/830466920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f355400fda0 con 0x7f3564109a80 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.461+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 msgr2=0x7f354c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.461+0000 7f3568f48700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 0x7f354c03ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3558006fd0 tx=0x7f3558006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 msgr2=0x7f3564109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3554009fb0 tx=0x7f3554011940 comp rx=0 tx=0).stop 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 shutdown_connections 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f354c039560 0x7f354c03ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 --2- 172.21.15.37:0/830466920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3564109a80 0x7f3564109300 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 >> 172.21.15.37:0/830466920 conn(0x7f35641034f0 msgr2=0x7f3564103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 shutdown_connections 2024-04-26T18:13:20.810 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:15.462+0000 7f3568f48700 1 -- 172.21.15.37:0/830466920 wait complete. 2024-04-26T18:13:20.811 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-04-26T18:13:21.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:21 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2855823585' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:13:22.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:13:22.020 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.215+0000 7fafd78e7700 1 Processor -- start 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.215+0000 7fafd78e7700 1 -- start start 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.215+0000 7fafd78e7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.215+0000 7fafd78e7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fafd007c7d0 con 0x7fafd007dda0 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafd5683700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafd5683700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52942/0 (socket says 172.21.15.37:52942) 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafd5683700 1 -- 172.21.15.37:0/790117357 learned_addr learned my addr 172.21.15.37:0/790117357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafd5683700 1 -- 172.21.15.37:0/790117357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd007c950 con 0x7fafd007dda0 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafd5683700 1 --2- 172.21.15.37:0/790117357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009e80 tx=0x7fafc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d356a8c894f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:22.021 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.216+0000 7fafcffff700 1 -- 172.21.15.37:0/790117357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc8017070 con 0x7fafd007dda0 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafcffff700 1 -- 172.21.15.37:0/790117357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafc800b8c0 con 0x7fafd007dda0 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafcffff700 1 -- 172.21.15.37:0/790117357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc8004490 con 0x7fafd007dda0 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 -- 172.21.15.37:0/790117357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 msgr2=0x7fafd007c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 --2- 172.21.15.37:0/790117357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009e80 tx=0x7fafc800b330 comp rx=0 tx=0).stop 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 -- 172.21.15.37:0/790117357 shutdown_connections 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 --2- 172.21.15.37:0/790117357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd007c200 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 -- 172.21.15.37:0/790117357 >> 172.21.15.37:0/790117357 conn(0x7fafd0104580 msgr2=0x7fafd01069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 -- 172.21.15.37:0/790117357 shutdown_connections 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.217+0000 7fafd78e7700 1 -- 172.21.15.37:0/790117357 wait complete. 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd78e7700 1 Processor -- start 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd78e7700 1 -- start start 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd78e7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.022 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd78e7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fafd010f550 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd5683700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd5683700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52958/0 (socket says 172.21.15.37:52958) 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd5683700 1 -- 172.21.15.37:0/2855823585 learned_addr learned my addr 172.21.15.37:0/2855823585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd5683700 1 -- 172.21.15.37:0/2855823585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019f650 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.218+0000 7fafd5683700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fafc8004a60 tx=0x7fafc800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc8020020 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafc8012430 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019f8e0 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc8017040 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a2580 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.219+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fafc8023020 con 0x7fafd007dda0 2024-04-26T18:13:22.023 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.220+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd0064a80 con 0x7fafd007dda0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.220+0000 7fafce7fc700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 0x7fafb803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.220+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fafc800f050 con 0x7fafd007dda0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.220+0000 7fafd4e82700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 0x7fafb803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.220+0000 7fafd4e82700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 0x7fafb803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.225+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafc802a020 con 0x7fafd007dda0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.431+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fafd01a2750 con 0x7fafd007dda0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.432+0000 7fafce7fc700 1 -- 172.21.15.37:0/2855823585 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fafd01a2750 con 0x7fafd007dda0 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 msgr2=0x7fafb803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 0x7fafb803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 msgr2=0x7fafd019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fafc8004a60 tx=0x7fafc800bd50 comp rx=0 tx=0).stop 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 shutdown_connections 2024-04-26T18:13:22.024 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7fafb80395b0 0x7fafb803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 --2- 172.21.15.37:0/2855823585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd007dda0 0x7fafd019f110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 >> 172.21.15.37:0/2855823585 conn(0x7fafd0104580 msgr2=0x7fafd0105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:22.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 shutdown_connections 2024-04-26T18:13:22.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:21.434+0000 7fafd78e7700 1 -- 172.21.15.37:0/2855823585 wait complete. 2024-04-26T18:13:22.025 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-04-26T18:13:22.939 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:22 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/801734221' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T18:13:22.939 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setuser ceph since I am not root 2024-04-26T18:13:22.939 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setgroup ceph since I am not root 2024-04-26T18:13:22.939 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:22.911+0000 7f18b5a5e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:13:23.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:23.102+0000 7f18b5a5e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:23.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:23.207+0000 7f18b5a5e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:13:23.823 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/801734221' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T18:13:23.823 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-mon[29171]: mgrmap e5: x(active, since 13s) 2024-04-26T18:13:23.823 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:23.528+0000 7f18b5a5e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:13:23.823 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:23.616+0000 7f18b5a5e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.404+0000 7f7fd8471700 1 Processor -- start 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.404+0000 7f7fd8471700 1 -- start start 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd8471700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd8471700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0109960 con 0x7f7fd0108f70 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52960/0 (socket says 172.21.15.37:52960) 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd620d700 1 -- 172.21.15.37:0/1216359590 learned_addr learned my addr 172.21.15.37:0/1216359590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd620d700 1 -- 172.21.15.37:0/1216359590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd010a170 con 0x7f7fd0108f70 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.405+0000 7f7fd620d700 1 --2- 172.21.15.37:0/1216359590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0098a0 tx=0x7f7fcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9929614e7430e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd520b700 1 -- 172.21.15.37:0/1216359590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fcc004160 con 0x7f7fd0108f70 2024-04-26T18:13:23.831 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd520b700 1 -- 172.21.15.37:0/1216359590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7fcc004300 con 0x7f7fd0108f70 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd520b700 1 -- 172.21.15.37:0/1216359590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fcc011af0 con 0x7f7fd0108f70 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 -- 172.21.15.37:0/1216359590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 msgr2=0x7f7fd0109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 --2- 172.21.15.37:0/1216359590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0098a0 tx=0x7f7fcc009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 -- 172.21.15.37:0/1216359590 shutdown_connections 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 --2- 172.21.15.37:0/1216359590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd0109390 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 -- 172.21.15.37:0/1216359590 >> 172.21.15.37:0/1216359590 conn(0x7f7fd01044f0 msgr2=0x7f7fd0106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 -- 172.21.15.37:0/1216359590 shutdown_connections 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.406+0000 7f7fd8471700 1 -- 172.21.15.37:0/1216359590 wait complete. 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd8471700 1 Processor -- start 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd8471700 1 -- start start 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd8471700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd8471700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010d520 con 0x7f7fd0108f70 2024-04-26T18:13:23.832 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52964/0 (socket says 172.21.15.37:52964) 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.407+0000 7f7fd620d700 1 -- 172.21.15.37:0/801734221 learned_addr learned my addr 172.21.15.37:0/801734221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fd620d700 1 -- 172.21.15.37:0/801734221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd007f050 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fd620d700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000c00 tx=0x7f7fcc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fcc003b20 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7fcc01a8c0 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd007d290 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fcc024940 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.408+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd007d770 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.409+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7fcc022020 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.409+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd0064a80 con 0x7f7fd0108f70 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.409+0000 7f7fc37fe700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 0x7f7fbc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:23.833 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.409+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fcc04d420 con 0x7f7fd0108f70 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.409+0000 7f7fd5a0c700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 0x7f7fbc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.410+0000 7f7fd5a0c700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 0x7f7fbc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006fd0 tx=0x7f7fc4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.414+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fcc04d180 con 0x7f7fd0108f70 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.682+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f7fd007df60 con 0x7f7fd0108f70 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.733+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7fcc02d3f0 con 0x7f7fd0108f70 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.735+0000 7f7fc37fe700 1 -- 172.21.15.37:0/801734221 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7fcc00bce0 con 0x7f7fd0108f70 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 msgr2=0x7f7fbc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 0x7f7fbc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4006fd0 tx=0x7f7fc4006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 msgr2=0x7f7fd007eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc000c00 tx=0x7f7fcc011b70 comp rx=0 tx=0).stop 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 shutdown_connections 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7fbc0395b0 0x7f7fbc03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 --2- 172.21.15.37:0/801734221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fd0108f70 0x7f7fd007eb10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 >> 172.21.15.37:0/801734221 conn(0x7f7fd01044f0 msgr2=0x7f7fd0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:23.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 shutdown_connections 2024-04-26T18:13:23.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:22.736+0000 7f7fd8471700 1 -- 172.21.15.37:0/801734221 wait complete. 2024-04-26T18:13:25.049 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:24 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1582421442' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:13:25.049 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:24 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:24.872+0000 7f18b5a5e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:13:25.049 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:25.049+0000 7f18b5a5e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.277+0000 7f3e9febf700 1 Processor -- start 2024-04-26T18:13:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9febf700 1 -- start start 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9febf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9febf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9807ce20 con 0x7f3e981078b0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9dc5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9dc5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52976/0 (socket says 172.21.15.37:52976) 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9dc5b700 1 -- 172.21.15.37:0/1120768722 learned_addr learned my addr 172.21.15.37:0/1120768722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9dc5b700 1 -- 172.21.15.37:0/1120768722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e98108210 con 0x7f3e981078b0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.278+0000 7f3e9dc5b700 1 --2- 172.21.15.37:0/1120768722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9793ffbd0ca80aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9cc59700 1 -- 172.21.15.37:0/1120768722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94004160 con 0x7f3e981078b0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9cc59700 1 -- 172.21.15.37:0/1120768722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e94004300 con 0x7f3e981078b0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9cc59700 1 -- 172.21.15.37:0/1120768722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94011af0 con 0x7f3e981078b0 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 -- 172.21.15.37:0/1120768722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 msgr2=0x7f3e98107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1120768722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098a0 tx=0x7f3e94009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.155 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 -- 172.21.15.37:0/1120768722 shutdown_connections 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1120768722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e98107cd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 -- 172.21.15.37:0/1120768722 >> 172.21.15.37:0/1120768722 conn(0x7f3e98103490 msgr2=0x7f3e981058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 -- 172.21.15.37:0/1120768722 shutdown_connections 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.279+0000 7f3e9febf700 1 -- 172.21.15.37:0/1120768722 wait complete. 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9febf700 1 Processor -- start 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9febf700 1 -- start start 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9febf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9febf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e98109240 con 0x7f3e981078b0 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9dc5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9dc5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52982/0 (socket says 172.21.15.37:52982) 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.280+0000 7f3e9dc5b700 1 -- 172.21.15.37:0/1582421442 learned_addr learned my addr 172.21.15.37:0/1582421442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e9dc5b700 1 -- 172.21.15.37:0/1582421442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9819f680 con 0x7f3e981078b0 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e9dc5b700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fb0 tx=0x7f3e94011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.156 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e940036a0 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e9401a430 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9819f910 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e981a2600 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.281+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e94019550 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f3e94019800 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e8affd700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 0x7f3e8403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e9404e850 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e9d45a700 1 -- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 msgr2=0x7f3e8403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e9d45a700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 0x7f3e8403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.282+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c005360 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.287+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e9400f2e0 con 0x7f3e981078b0 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.483+0000 7f3e9d45a700 1 -- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 msgr2=0x7f3e8403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:25.157 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.483+0000 7f3e9d45a700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 0x7f3e8403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.558+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3e7c005e60 con 0x7f3e981078b0 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.559+0000 7f3e8affd700 1 -- 172.21.15.37:0/1582421442 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3e9404e270 con 0x7f3e981078b0 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 msgr2=0x7f3e8403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 0x7f3e8403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 msgr2=0x7f3e9819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fb0 tx=0x7f3e94011940 comp rx=0 tx=0).stop 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 shutdown_connections 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f3e840395b0 0x7f3e8403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 --2- 172.21.15.37:0/1582421442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e981078b0 0x7f3e9819f140 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 >> 172.21.15.37:0/1582421442 conn(0x7f3e98103490 msgr2=0x7f3e98195d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 shutdown_connections 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:24.561+0000 7f3e9febf700 1 -- 172.21.15.37:0/1582421442 wait complete. 2024-04-26T18:13:25.158 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-26T18:13:25.159 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-04-26T18:13:25.365 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:25.237+0000 7f18b5a5e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:13:25.639 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:25.540+0000 7f18b5a5e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:13:25.640 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:25.639+0000 7f18b5a5e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:13:25.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:25.852+0000 7f18b5a5e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:13:26.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:26.052+0000 7f18b5a5e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:26.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:26.675+0000 7f18b5a5e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:13:27.421 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:27.112+0000 7f18b5a5e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:13:27.421 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:27.212+0000 7f18b5a5e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:13:27.421 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:27.420+0000 7f18b5a5e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:13:28.232 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:27.964+0000 7f18b5a5e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:13:28.232 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.059+0000 7f18b5a5e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:13:28.586 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.309+0000 7f18b5a5e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:13:28.586 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.446+0000 7f18b5a5e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:13:28.586 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.545+0000 7f18b5a5e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:13:28.978 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.666+0000 7f18b5a5e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:28.978 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:28.978+0000 7f18b5a5e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:13:29.880 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:29.603+0000 7f18b5a5e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:13:29.881 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:29.827+0000 7f18b5a5e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:13:30.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: Active manager daemon x restarted 2024-04-26T18:13:30.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: Activating manager daemon x 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: mgrmap e6: x(active, starting, since 0.00355211s) 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: Manager daemon x is now available 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:29 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: Found migration_current of "None". Setting to last migration. 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:30] ENGINE Bus STARTING 2024-04-26T18:13:31.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:30 smithi037 ceph-mon[29171]: mgrmap e7: x(active, since 1.00571s) 2024-04-26T18:13:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7efd5b0700 1 Processor -- start 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7efd5b0700 1 -- start start 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7efd5b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7efd5b0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef807d8e0 con 0x7f7ef807eeb0 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7ef6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7ef6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44402/0 (socket says 172.21.15.37:44402) 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.576+0000 7f7ef6ffd700 1 -- 172.21.15.37:0/2299278711 learned_addr learned my addr 172.21.15.37:0/2299278711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7ef6ffd700 1 -- 172.21.15.37:0/2299278711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef807da60 con 0x7f7ef807eeb0 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7ef6ffd700 1 --2- 172.21.15.37:0/2299278711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098a0 tx=0x7f7ee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336008b5fe385e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7ef5ffb700 1 -- 172.21.15.37:0/2299278711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee8004160 con 0x7f7ef807eeb0 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7ef5ffb700 1 -- 172.21.15.37:0/2299278711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ee8004300 con 0x7f7ef807eeb0 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7ef5ffb700 1 -- 172.21.15.37:0/2299278711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee8011af0 con 0x7f7ef807eeb0 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7efd5b0700 1 -- 172.21.15.37:0/2299278711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 msgr2=0x7f7ef807d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.577+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/2299278711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80098a0 tx=0x7f7ee8009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 -- 172.21.15.37:0/2299278711 shutdown_connections 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/2299278711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef807d310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 -- 172.21.15.37:0/2299278711 >> 172.21.15.37:0/2299278711 conn(0x7f7ef81045a0 msgr2=0x7f7ef81069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 -- 172.21.15.37:0/2299278711 shutdown_connections 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 -- 172.21.15.37:0/2299278711 wait complete. 2024-04-26T18:13:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.578+0000 7f7efd5b0700 1 Processor -- start 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7efd5b0700 1 -- start start 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7efd5b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7efd5b0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef810f570 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7ef6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7ef6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44414/0 (socket says 172.21.15.37:44414) 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7ef6ffd700 1 -- 172.21.15.37:0/1951149664 learned_addr learned my addr 172.21.15.37:0/1951149664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7ef6ffd700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef819f690 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.579+0000 7f7ef6ffd700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fb0 tx=0x7f7ee8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.580+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee80036a0 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.580+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ee801a430 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.580+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef819f920 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.580+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee8019550 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.580+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a2610 con 0x7f7ef807eeb0 2024-04-26T18:13:31.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7ee8019800 con 0x7f7ef807eeb0 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7edffff700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ee003c180 con 0x7f7ee00395b0 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ee804d8a0 con 0x7f7ef807eeb0 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7ef67fc700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.581+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.782+0000 7f7ef67fc700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:25.782+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:26.183+0000 7f7ef67fc700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:26.183+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:26.984+0000 7f7ef67fc700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:26.984+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:28.585+0000 7f7ef67fc700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/199318008 2024-04-26T18:13:31.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:28.585+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:29.831+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7ee8019b40 con 0x7f7ef807eeb0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:29.831+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 msgr2=0x7f7ee003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:29.831+0000 7f7edffff700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.832+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7ee804fa10 con 0x7f7ef807eeb0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.833+0000 7f7edffff700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 0x7f7ee003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.833+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7ee003c180 con 0x7f7ee003cea0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.833+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 0x7f7ee003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.833+0000 7f7ef67fc700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 0x7f7ee003f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7eec003a10 tx=0x7f7eec0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.834+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7ee003c180 con 0x7f7ee003cea0 2024-04-26T18:13:31.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7ef8064a80 con 0x7f7ee003cea0 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7edffff700 1 -- 172.21.15.37:0/1951149664 <== mgr.14120 v2:172.21.15.37:6800/1233564213 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7ef8064a80 con 0x7f7ee003cea0 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 msgr2=0x7f7ee003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 0x7f7ee003f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7eec003a10 tx=0x7f7eec0092b0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 msgr2=0x7f7ef819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fb0 tx=0x7f7ee8011940 comp rx=0 tx=0).stop 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 shutdown_connections 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7ee003cea0 0x7f7ee003f290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ef807eeb0 0x7f7ef819f150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 --2- 172.21.15.37:0/1951149664 >> [v2:172.21.15.37:6800/199318008,v1:172.21.15.37:6801/199318008] conn(0x7f7ee00395b0 0x7f7ee003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.843+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 >> 172.21.15.37:0/1951149664 conn(0x7f7ef81045a0 msgr2=0x7f7ef81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.844+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 shutdown_connections 2024-04-26T18:13:31.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:30.844+0000 7f7efd5b0700 1 -- 172.21.15.37:0/1951149664 wait complete. 2024-04-26T18:13:31.411 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-04-26T18:13:31.411 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-04-26T18:13:31.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:30] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T18:13:31.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:13:31.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:13:31.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:30] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T18:13:31.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:30] ENGINE Bus STARTED 2024-04-26T18:13:31.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:31 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:32.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f2431e6e700 1 Processor -- start 2024-04-26T18:13:32.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f2431e6e700 1 -- start start 2024-04-26T18:13:32.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f2431e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f2431e6e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f242c07ce20 con 0x7f242c107850 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44484/0 (socket says 172.21.15.37:44484) 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f242b7fe700 1 -- 172.21.15.37:0/3819934816 learned_addr learned my addr 172.21.15.37:0/3819934816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.801+0000 7f242b7fe700 1 -- 172.21.15.37:0/3819934816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c1081b0 con 0x7f242c107850 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f242b7fe700 1 --2- 172.21.15.37:0/3819934816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098a0 tx=0x7f241c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934996223b1b7019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f242a7fc700 1 -- 172.21.15.37:0/3819934816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c004160 con 0x7f242c107850 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f242a7fc700 1 -- 172.21.15.37:0/3819934816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f241c004300 con 0x7f242c107850 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f242a7fc700 1 -- 172.21.15.37:0/3819934816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c011af0 con 0x7f242c107850 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 -- 172.21.15.37:0/3819934816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 msgr2=0x7f242c107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 --2- 172.21.15.37:0/3819934816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098a0 tx=0x7f241c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 -- 172.21.15.37:0/3819934816 shutdown_connections 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 --2- 172.21.15.37:0/3819934816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c107c70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:32.608 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 -- 172.21.15.37:0/3819934816 >> 172.21.15.37:0/3819934816 conn(0x7f242c103410 msgr2=0x7f242c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.802+0000 7f2431e6e700 1 -- 172.21.15.37:0/3819934816 shutdown_connections 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f2431e6e700 1 -- 172.21.15.37:0/3819934816 wait complete. 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f2431e6e700 1 Processor -- start 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f2431e6e700 1 -- start start 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f2431e6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f2431e6e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f242c1091e0 con 0x7f242c107850 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.803+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f242b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44498/0 (socket says 172.21.15.37:44498) 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f242b7fe700 1 -- 172.21.15.37:0/206258132 learned_addr learned my addr 172.21.15.37:0/206258132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f242b7fe700 1 -- 172.21.15.37:0/206258132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c19f670 con 0x7f242c107850 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f242b7fe700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f241c009fb0 tx=0x7f241c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c0036a0 con 0x7f242c107850 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c19f900 con 0x7f242c107850 2024-04-26T18:13:32.609 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f241c01a430 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c019550 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.804+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1a25f0 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.805+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f241c019800 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.805+0000 7f2428ff9700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 0x7f241403b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.805+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c064a80 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.805+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f241c04d870 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.806+0000 7f242affd700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 0x7f241403b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.806+0000 7f242affd700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 0x7f241403b940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2420006fd0 tx=0x7f2420006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.811+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f241c01a5e0 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:31.883+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f241c02d960 con 0x7f242c107850 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.040+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f242c02e800 con 0x7f2414039480 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.045+0000 7f2428ff9700 1 -- 172.21.15.37:0/206258132 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f242c02e800 con 0x7f2414039480 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 msgr2=0x7f241403b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:32.610 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 0x7f241403b940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2420006fd0 tx=0x7f2420006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 msgr2=0x7f242c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f241c009fb0 tx=0x7f241c011940 comp rx=0 tx=0).stop 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 shutdown_connections 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f2414039480 0x7f241403b940 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 --2- 172.21.15.37:0/206258132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f242c107850 0x7f242c19f130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 >> 172.21.15.37:0/206258132 conn(0x7f242c103410 msgr2=0x7f242c195d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 shutdown_connections 2024-04-26T18:13:32.611 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:32.047+0000 7f2431e6e700 1 -- 172.21.15.37:0/206258132 wait complete. 2024-04-26T18:13:33.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:32 smithi037 ceph-mon[29171]: mgrmap e8: x(active, since 2s) 2024-04-26T18:13:33.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:32 smithi037 ceph-mon[29171]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:33.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:32 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:33.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:32 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:33.813 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T18:13:33.813 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e29038700 1 Processor -- start 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e29038700 1 -- start start 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e29038700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e29038700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2407d8a0 con 0x7f4e2407ee70 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44514/0 (socket says 172.21.15.37:44514) 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e22d9d700 1 -- 172.21.15.37:0/238227095 learned_addr learned my addr 172.21.15.37:0/238227095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.016+0000 7f4e22d9d700 1 -- 172.21.15.37:0/238227095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2407da20 con 0x7f4e2407ee70 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e22d9d700 1 --2- 172.21.15.37:0/238227095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e140098a0 tx=0x7f4e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0abbc7a533132f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e21d9b700 1 -- 172.21.15.37:0/238227095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e14004160 con 0x7f4e2407ee70 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e21d9b700 1 -- 172.21.15.37:0/238227095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e14004300 con 0x7f4e2407ee70 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e21d9b700 1 -- 172.21.15.37:0/238227095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e14011af0 con 0x7f4e2407ee70 2024-04-26T18:13:33.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e29038700 1 -- 172.21.15.37:0/238227095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 msgr2=0x7f4e2407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e29038700 1 --2- 172.21.15.37:0/238227095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e140098a0 tx=0x7f4e14009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e29038700 1 -- 172.21.15.37:0/238227095 shutdown_connections 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e29038700 1 --2- 172.21.15.37:0/238227095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2407d2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.017+0000 7f4e29038700 1 -- 172.21.15.37:0/238227095 >> 172.21.15.37:0/238227095 conn(0x7f4e241044f0 msgr2=0x7f4e24106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.018+0000 7f4e29038700 1 -- 172.21.15.37:0/238227095 shutdown_connections 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.018+0000 7f4e29038700 1 -- 172.21.15.37:0/238227095 wait complete. 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.018+0000 7f4e29038700 1 Processor -- start 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e29038700 1 -- start start 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e29038700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e29038700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2410f540 con 0x7f4e2407ee70 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e22d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44518/0 (socket says 172.21.15.37:44518) 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e22d9d700 1 -- 172.21.15.37:0/252235856 learned_addr learned my addr 172.21.15.37:0/252235856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:33.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e22d9d700 1 -- 172.21.15.37:0/252235856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2419f5b0 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e22d9d700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e14003f40 tx=0x7f4e14011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.019+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e140038b0 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.020+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2419f840 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.020+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e14003a50 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.020+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e1401a5c0 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.020+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a2530 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.021+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4e1401a8c0 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.021+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24064a80 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.021+0000 7f4e0bfff700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 0x7f4e0c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.021+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e1404db30 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.021+0000 7f4e2259c700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 0x7f4e0c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.022+0000 7f4e2259c700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 0x7f4e0c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006fd0 tx=0x7f4e18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.026+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e14019c70 con 0x7f4e2407ee70 2024-04-26T18:13:33.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.241+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4e241057a0 con 0x7f4e0c039560 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.242+0000 7f4e0bfff700 1 -- 172.21.15.37:0/252235856 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4e241057a0 con 0x7f4e0c039560 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 msgr2=0x7f4e0c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 0x7f4e0c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e18006fd0 tx=0x7f4e18006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 msgr2=0x7f4e2419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e14003f40 tx=0x7f4e14011890 comp rx=0 tx=0).stop 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 shutdown_connections 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4e0c039560 0x7f4e0c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 --2- 172.21.15.37:0/252235856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e2407ee70 0x7f4e2419f070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 >> 172.21.15.37:0/252235856 conn(0x7f4e241044f0 msgr2=0x7f4e24105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 shutdown_connections 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:33.244+0000 7f4e29038700 1 -- 172.21.15.37:0/252235856 wait complete. 2024-04-26T18:13:33.817 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-04-26T18:13:34.124 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:33 smithi037 ceph-mon[29171]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: Generating public/private rsa key pair. 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: Your identification has been saved in /tmp/tmptl5eygoo/key. 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: Your public key has been saved in /tmp/tmptl5eygoo/key.pub. 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: The key fingerprint is: 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: SHA256:8qEcQi1DI8bIl2Wt5eLt0djbY/ZzFy3TRo0oJjAVzKc ceph-755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: The key's randomart image is: 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: +---[RSA 3072]----+ 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: |.oo =o.+o. | 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: |.o.=.oooo . | 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | . + =o o . ..| 2024-04-26T18:13:34.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | . = .E o . . o| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | o = So . o.| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | + O + o.+| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | + o o +.| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | . . = . ..| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: | o o.o .| 2024-04-26T18:13:34.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: +----[SHA256]-----+ 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8eacbe3700 1 Processor -- start 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8eacbe3700 1 -- start start 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8eacbe3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8eacbe3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8109960 con 0x7f8ea8108f70 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44528/0 (socket says 172.21.15.37:44528) 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8ea659c700 1 -- 172.21.15.37:0/1846321724 learned_addr learned my addr 172.21.15.37:0/1846321724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:35.278 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.236+0000 7f8ea659c700 1 -- 172.21.15.37:0/1846321724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea810a170 con 0x7f8ea8108f70 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8ea659c700 1 --2- 172.21.15.37:0/1846321724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e980098a0 tx=0x7f8e98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed77b51f664d3f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8ea559a700 1 -- 172.21.15.37:0/1846321724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e98004160 con 0x7f8ea8108f70 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8ea559a700 1 -- 172.21.15.37:0/1846321724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e98004300 con 0x7f8ea8108f70 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8ea559a700 1 -- 172.21.15.37:0/1846321724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e98011af0 con 0x7f8ea8108f70 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8eacbe3700 1 -- 172.21.15.37:0/1846321724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 msgr2=0x7f8ea8109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.237+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/1846321724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e980098a0 tx=0x7f8e98009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 -- 172.21.15.37:0/1846321724 shutdown_connections 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/1846321724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea8109390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 -- 172.21.15.37:0/1846321724 >> 172.21.15.37:0/1846321724 conn(0x7f8ea81044f0 msgr2=0x7f8ea8106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 -- 172.21.15.37:0/1846321724 shutdown_connections 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 -- 172.21.15.37:0/1846321724 wait complete. 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.238+0000 7f8eacbe3700 1 Processor -- start 2024-04-26T18:13:35.279 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8eacbe3700 1 -- start start 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8eacbe3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8eacbe3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea810d520 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44542/0 (socket says 172.21.15.37:44542) 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8ea659c700 1 -- 172.21.15.37:0/4019860145 learned_addr learned my addr 172.21.15.37:0/4019860145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8ea659c700 1 -- 172.21.15.37:0/4019860145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea819d400 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8ea659c700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e98011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e98003b20 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea819d690 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.239+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e9801a8c0 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.240+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e98024940 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.240+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81a0380 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.240+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8e98022020 con 0x7f8ea8108f70 2024-04-26T18:13:35.280 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.240+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea8064a80 con 0x7f8ea8108f70 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.241+0000 7f8e977fe700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 0x7f8e9003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.241+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e9804da40 con 0x7f8ea8108f70 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.241+0000 7f8ea5d9b700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 0x7f8e9003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.241+0000 7f8ea5d9b700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 0x7f8e9003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006fd0 tx=0x7f8e9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.246+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e98021d80 con 0x7f8ea8108f70 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.459+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8ea8105760 con 0x7f8e900395b0 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.731+0000 7f8e977fe700 1 -- 172.21.15.37:0/4019860145 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ea8105760 con 0x7f8e900395b0 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 msgr2=0x7f8e9003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 0x7f8e9003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c006fd0 tx=0x7f8e9c006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 msgr2=0x7f8ea819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e98011b70 comp rx=0 tx=0).stop 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 shutdown_connections 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f8e900395b0 0x7f8e9003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:35.281 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 --2- 172.21.15.37:0/4019860145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ea8108f70 0x7f8ea819cec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:35.282 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 >> 172.21.15.37:0/4019860145 conn(0x7f8ea81044f0 msgr2=0x7f8ea8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:35.282 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 shutdown_connections 2024-04-26T18:13:35.282 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:34.733+0000 7f8eacbe3700 1 -- 172.21.15.37:0/4019860145 wait complete. 2024-04-26T18:13:35.904 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:35 smithi037 ceph-mon[29171]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:35.904 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:35 smithi037 ceph-mon[29171]: Generating ssh key... 2024-04-26T18:13:35.904 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:35 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:35.904 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:35 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.679+0000 7f721ccf9700 1 Processor -- start 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.679+0000 7f721ccf9700 1 -- start start 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721ccf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721ccf9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7218076d20 con 0x7f7218109a80 2024-04-26T18:13:36.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40014/0 (socket says 172.21.15.37:40014) 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721659c700 1 -- 172.21.15.37:0/3932039086 learned_addr learned my addr 172.21.15.37:0/3932039086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721659c700 1 -- 172.21.15.37:0/3932039086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721810c3b0 con 0x7f7218109a80 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721659c700 1 --2- 172.21.15.37:0/3932039086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f7208009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cfed90a8aced87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.680+0000 7f721559a700 1 -- 172.21.15.37:0/3932039086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208004160 con 0x7f7218109a80 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721559a700 1 -- 172.21.15.37:0/3932039086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7208004300 con 0x7f7218109a80 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721559a700 1 -- 172.21.15.37:0/3932039086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208011af0 con 0x7f7218109a80 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 -- 172.21.15.37:0/3932039086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 msgr2=0x7f721810be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 --2- 172.21.15.37:0/3932039086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f7208009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 -- 172.21.15.37:0/3932039086 shutdown_connections 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 --2- 172.21.15.37:0/3932039086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721810be70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 -- 172.21.15.37:0/3932039086 >> 172.21.15.37:0/3932039086 conn(0x7f72181034f0 msgr2=0x7f7218105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 -- 172.21.15.37:0/3932039086 shutdown_connections 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.681+0000 7f721ccf9700 1 -- 172.21.15.37:0/3932039086 wait complete. 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721ccf9700 1 Processor -- start 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721ccf9700 1 -- start start 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721ccf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721ccf9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f721810d320 con 0x7f7218109a80 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40024/0 (socket says 172.21.15.37:40024) 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.682+0000 7f721659c700 1 -- 172.21.15.37:0/4186054065 learned_addr learned my addr 172.21.15.37:0/4186054065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f721659c700 1 -- 172.21.15.37:0/4186054065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721819f650 con 0x7f7218109a80 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f721659c700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7208011ac0 tx=0x7f7208011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208003970 con 0x7f7218109a80 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7208003b10 con 0x7f7218109a80 2024-04-26T18:13:36.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721819f8e0 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208024610 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.683+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72181a25d0 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.684+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7208024840 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.684+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7218064a80 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.684+0000 7f72077fe700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 0x7f720003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.684+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f720804d8d0 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.684+0000 7f7215d9b700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 0x7f720003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.685+0000 7f7215d9b700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 0x7f720003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.689+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72080172b0 con 0x7f7218109a80 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.904+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f72181046b0 con 0x7f7200039600 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.904+0000 7f72077fe700 1 -- 172.21.15.37:0/4186054065 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f72181046b0 con 0x7f7200039600 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 msgr2=0x7f720003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 0x7f720003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:36.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 msgr2=0x7f721819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7208011ac0 tx=0x7f7208011ba0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 shutdown_connections 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f7200039600 0x7f720003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 --2- 172.21.15.37:0/4186054065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7218109a80 0x7f721819f110 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.906+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 >> 172.21.15.37:0/4186054065 conn(0x7f72181034f0 msgr2=0x7f7218103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.907+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 shutdown_connections 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:35.907+0000 7f721ccf9700 1 -- 172.21.15.37:0/4186054065 wait complete. 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T18:13:36.505 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-04-26T18:13:36.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:36 smithi037 ceph-mon[29171]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:38.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:37 smithi037 ceph-mon[29171]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:38.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:37 smithi037 ceph-mon[29171]: Deploying cephadm binary to smithi037 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.942+0000 7f9562c05700 1 Processor -- start 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.942+0000 7f9562c05700 1 -- start start 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f9562c05700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f9562c05700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f955c07d8e0 con 0x7f955c07eeb0 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f95609a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f95609a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40028/0 (socket says 172.21.15.37:40028) 2024-04-26T18:13:41.991 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f95609a1700 1 -- 172.21.15.37:0/3311105978 learned_addr learned my addr 172.21.15.37:0/3311105978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f95609a1700 1 -- 172.21.15.37:0/3311105978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c07da60 con 0x7f955c07eeb0 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f95609a1700 1 --2- 172.21.15.37:0/3311105978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95500098a0 tx=0x7f9550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce77cd3d004a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f955b7fe700 1 -- 172.21.15.37:0/3311105978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9550004160 con 0x7f955c07eeb0 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f955b7fe700 1 -- 172.21.15.37:0/3311105978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9550004300 con 0x7f955c07eeb0 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.943+0000 7f955b7fe700 1 -- 172.21.15.37:0/3311105978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9550011af0 con 0x7f955c07eeb0 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 -- 172.21.15.37:0/3311105978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 msgr2=0x7f955c07d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 --2- 172.21.15.37:0/3311105978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95500098a0 tx=0x7f9550009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 -- 172.21.15.37:0/3311105978 shutdown_connections 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 --2- 172.21.15.37:0/3311105978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c07d310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 -- 172.21.15.37:0/3311105978 >> 172.21.15.37:0/3311105978 conn(0x7f955c1044f0 msgr2=0x7f955c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 -- 172.21.15.37:0/3311105978 shutdown_connections 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.944+0000 7f9562c05700 1 -- 172.21.15.37:0/3311105978 wait complete. 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f9562c05700 1 Processor -- start 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f9562c05700 1 -- start start 2024-04-26T18:13:41.992 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f9562c05700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f9562c05700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10f500 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f95609a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f95609a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40034/0 (socket says 172.21.15.37:40034) 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f95609a1700 1 -- 172.21.15.37:0/2660529198 learned_addr learned my addr 172.21.15.37:0/2660529198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.945+0000 7f95609a1700 1 -- 172.21.15.37:0/2660529198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c19f600 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f95609a1700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9550003f40 tx=0x7f9550011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95500038b0 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c19f890 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9550003a50 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f955001a5c0 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.946+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1a2580 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.947+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f955001a8c0 con 0x7f955c07eeb0 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.947+0000 7f9559ffb700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 0x7f954c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.947+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f955004db00 con 0x7f955c07eeb0 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.947+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c064a80 con 0x7f955c07eeb0 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.947+0000 7f955bfff700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 0x7f954c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.948+0000 7f955bfff700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 0x7f954c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9548006fd0 tx=0x7f9548006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:36.952+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9550019c70 con 0x7f955c07eeb0 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:37.164+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7f955c105790 con 0x7f954c039560 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.417+0000 7f9559ffb700 1 -- 172.21.15.37:0/2660529198 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f955c105790 con 0x7f954c039560 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.419+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 msgr2=0x7f954c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 0x7f954c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9548006fd0 tx=0x7f9548006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 msgr2=0x7f955c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9550003f40 tx=0x7f9550011890 comp rx=0 tx=0).stop 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 shutdown_connections 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f954c039560 0x7f954c03ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:41.994 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 --2- 172.21.15.37:0/2660529198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f955c07eeb0 0x7f955c19f0c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:41.995 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 >> 172.21.15.37:0/2660529198 conn(0x7f955c1044f0 msgr2=0x7f955c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:41.995 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 shutdown_connections 2024-04-26T18:13:41.995 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:41.420+0000 7f9562c05700 1 -- 172.21.15.37:0/2660529198 wait complete. 2024-04-26T18:13:41.995 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-04-26T18:13:42.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:42 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:42.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:42 smithi037 ceph-mon[29171]: Added host smithi037 2024-04-26T18:13:42.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:42 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:43.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T18:13:43.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.383+0000 7f12e68b4700 1 Processor -- start 2024-04-26T18:13:43.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.383+0000 7f12e68b4700 1 -- start start 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.383+0000 7f12e68b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.383+0000 7f12e68b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01087f0 con 0x7f12e0107e90 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12dffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12dffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40050/0 (socket says 172.21.15.37:40050) 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12dffff700 1 -- 172.21.15.37:0/3842716555 learned_addr learned my addr 172.21.15.37:0/3842716555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12dffff700 1 -- 172.21.15.37:0/3842716555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e0108ff0 con 0x7f12e0107e90 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12dffff700 1 --2- 172.21.15.37:0/3842716555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12d000bb40 tx=0x7f12d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa796095638527c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12deffd700 1 -- 172.21.15.37:0/3842716555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d000dbe0 con 0x7f12e0107e90 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12deffd700 1 -- 172.21.15.37:0/3842716555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f12d00044c0 con 0x7f12e0107e90 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.384+0000 7f12deffd700 1 -- 172.21.15.37:0/3842716555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d0010470 con 0x7f12e0107e90 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 -- 172.21.15.37:0/3842716555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 msgr2=0x7f12e01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:43.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 --2- 172.21.15.37:0/3842716555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12d000bb40 tx=0x7f12d000be50 comp rx=0 tx=0).stop 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 -- 172.21.15.37:0/3842716555 shutdown_connections 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 --2- 172.21.15.37:0/3842716555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e01082b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 -- 172.21.15.37:0/3842716555 >> 172.21.15.37:0/3842716555 conn(0x7f12e0103450 msgr2=0x7f12e0105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 -- 172.21.15.37:0/3842716555 shutdown_connections 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.385+0000 7f12e68b4700 1 -- 172.21.15.37:0/3842716555 wait complete. 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12e68b4700 1 Processor -- start 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12e68b4700 1 -- start start 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12e68b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12e68b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12e007bfc0 con 0x7f12e0107e90 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12dffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12dffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40064/0 (socket says 172.21.15.37:40064) 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12dffff700 1 -- 172.21.15.37:0/812064836 learned_addr learned my addr 172.21.15.37:0/812064836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:43.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.386+0000 7f12dffff700 1 -- 172.21.15.37:0/812064836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e019b160 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12dffff700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12d000b340 tx=0x7f12d0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d0027070 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e019b3f0 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f12d0010bb0 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12d0022ac0 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.387+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e019e0e0 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.388+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f12d002a070 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.388+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e0064a80 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.388+0000 7f12dd7fa700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 0x7f12c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.388+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f12d001f070 con 0x7f12e0107e90 2024-04-26T18:13:43.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.388+0000 7f12df7fe700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 0x7f12c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.389+0000 7f12df7fe700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 0x7f12c803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006fd0 tx=0x7f12d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.393+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12d0015070 con 0x7f12e0107e90 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.636+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f12e0063860 con 0x7f12c80395b0 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.644+0000 7f12dd7fa700 1 -- 172.21.15.37:0/812064836 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f12e0063860 con 0x7f12c80395b0 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 msgr2=0x7f12c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 0x7f12c803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f12d4006fd0 tx=0x7f12d4006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 msgr2=0x7f12e019ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12d000b340 tx=0x7f12d0003dc0 comp rx=0 tx=0).stop 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 shutdown_connections 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f12c80395b0 0x7f12c803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 --2- 172.21.15.37:0/812064836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12e0107e90 0x7f12e019ac20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 >> 172.21.15.37:0/812064836 conn(0x7f12e0103450 msgr2=0x7f12e01911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 shutdown_connections 2024-04-26T18:13:43.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:42.646+0000 7f12e68b4700 1 -- 172.21.15.37:0/812064836 wait complete. 2024-04-26T18:13:43.346 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-04-26T18:13:43.918 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:43 smithi037 ceph-mon[29171]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:43.918 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:43 smithi037 ceph-mon[29171]: Saving service mon spec with placement count:5 2024-04-26T18:13:43.918 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:43 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:45.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:45 smithi037 ceph-mon[29171]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:45.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:45 smithi037 ceph-mon[29171]: Saving service mgr spec with placement count:2 2024-04-26T18:13:45.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:45 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:45.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.804+0000 7f756c104700 1 Processor -- start 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.804+0000 7f756c104700 1 -- start start 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.804+0000 7f756c104700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.804+0000 7f756c104700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7564109870 con 0x7f7564108ef0 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7569ea0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7569ea0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40066/0 (socket says 172.21.15.37:40066) 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7569ea0700 1 -- 172.21.15.37:0/4240058728 learned_addr learned my addr 172.21.15.37:0/4240058728 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7569ea0700 1 -- 172.21.15.37:0/4240058728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756410a0d0 con 0x7f7564108ef0 2024-04-26T18:13:45.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7569ea0700 1 --2- 172.21.15.37:0/4240058728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7554009e80 tx=0x7f755400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8115915f59dc28a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7568e9e700 1 -- 172.21.15.37:0/4240058728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7554017070 con 0x7f7564108ef0 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7568e9e700 1 -- 172.21.15.37:0/4240058728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f755400b8c0 con 0x7f7564108ef0 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.805+0000 7f7568e9e700 1 -- 172.21.15.37:0/4240058728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7554004490 con 0x7f7564108ef0 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 -- 172.21.15.37:0/4240058728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 msgr2=0x7f7564109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 --2- 172.21.15.37:0/4240058728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7554009e80 tx=0x7f755400b330 comp rx=0 tx=0).stop 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 -- 172.21.15.37:0/4240058728 shutdown_connections 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 --2- 172.21.15.37:0/4240058728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f7564109330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 -- 172.21.15.37:0/4240058728 >> 172.21.15.37:0/4240058728 conn(0x7f75641044c0 msgr2=0x7f7564106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 -- 172.21.15.37:0/4240058728 shutdown_connections 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.806+0000 7f756c104700 1 -- 172.21.15.37:0/4240058728 wait complete. 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f756c104700 1 Processor -- start 2024-04-26T18:13:45.447 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f756c104700 1 -- start start 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f756c104700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f756c104700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f756410af60 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f7569ea0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f7569ea0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40080/0 (socket says 172.21.15.37:40080) 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f7569ea0700 1 -- 172.21.15.37:0/4261990995 learned_addr learned my addr 172.21.15.37:0/4261990995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f7569ea0700 1 -- 172.21.15.37:0/4261990995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756407f0a0 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.807+0000 7f7569ea0700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f755400b7b0 tx=0x7f755400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.808+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7554020020 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.808+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7554012430 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.808+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756407d290 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.808+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7554017040 con 0x7f7564108ef0 2024-04-26T18:13:45.448 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.808+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756407d7d0 con 0x7f7564108ef0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7554023020 con 0x7f7564108ef0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7564064a80 con 0x7f7564108ef0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f7562ffd700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 0x7f754c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f755400f050 con 0x7f7564108ef0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f756969f700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 0x7f754c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.809+0000 7f756969f700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 0x7f754c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:43.814+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f755402a020 con 0x7f7564108ef0 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.040+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7564105720 con 0x7f754c039560 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.047+0000 7f7562ffd700 1 -- 172.21.15.37:0/4261990995 <== mgr.14120 v2:172.21.15.37:6800/1233564213 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7564105720 con 0x7f754c039560 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 msgr2=0x7f754c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 0x7f754c03ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7558006fd0 tx=0x7f7558006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:45.449 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 msgr2=0x7f756407eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f755400b7b0 tx=0x7f755400be20 comp rx=0 tx=0).stop 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 shutdown_connections 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f754c039560 0x7f754c03ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 --2- 172.21.15.37:0/4261990995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7564108ef0 0x7f756407eb60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.049+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 >> 172.21.15.37:0/4261990995 conn(0x7f75641044c0 msgr2=0x7f7564105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.050+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 shutdown_connections 2024-04-26T18:13:45.450 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:44.050+0000 7f756c104700 1 -- 172.21.15.37:0/4261990995 wait complete. 2024-04-26T18:13:46.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:46 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:46.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:46 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:46.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f647d71f700 1 Processor -- start 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f647d71f700 1 -- start start 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f647d71f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f647d71f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6478076d40 con 0x7f6478109a60 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f6476ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f6476ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43644/0 (socket says 172.21.15.37:43644) 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.899+0000 7f6476ffd700 1 -- 172.21.15.37:0/897727178 learned_addr learned my addr 172.21.15.37:0/897727178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.900+0000 7f6476ffd700 1 -- 172.21.15.37:0/897727178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647810c390 con 0x7f6478109a60 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.900+0000 7f6476ffd700 1 --2- 172.21.15.37:0/897727178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64680098a0 tx=0x7f6468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395ba4d5f9d7e5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.900+0000 7f6475ffb700 1 -- 172.21.15.37:0/897727178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468004160 con 0x7f6478109a60 2024-04-26T18:13:46.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.900+0000 7f6475ffb700 1 -- 172.21.15.37:0/897727178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6468004300 con 0x7f6478109a60 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.900+0000 7f6475ffb700 1 -- 172.21.15.37:0/897727178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468011a20 con 0x7f6478109a60 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 -- 172.21.15.37:0/897727178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 msgr2=0x7f647810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 --2- 172.21.15.37:0/897727178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64680098a0 tx=0x7f6468009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 -- 172.21.15.37:0/897727178 shutdown_connections 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 --2- 172.21.15.37:0/897727178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647810be50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 -- 172.21.15.37:0/897727178 >> 172.21.15.37:0/897727178 conn(0x7f64781034f0 msgr2=0x7f6478105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 -- 172.21.15.37:0/897727178 shutdown_connections 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 -- 172.21.15.37:0/897727178 wait complete. 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.901+0000 7f647d71f700 1 Processor -- start 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f647d71f700 1 -- start start 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f647d71f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f647d71f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f647810d390 con 0x7f6478109a60 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f6476ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:46.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f6476ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43646/0 (socket says 172.21.15.37:43646) 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f6476ffd700 1 -- 172.21.15.37:0/86147296 learned_addr learned my addr 172.21.15.37:0/86147296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f6476ffd700 1 -- 172.21.15.37:0/86147296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647819f5c0 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.902+0000 7f6476ffd700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64680116e0 tx=0x7f6468011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.903+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6468003b20 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.903+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647819f850 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.903+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64781a2540 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.903+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f646801b8f0 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.903+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64680238c0 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6468023b40 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6458005330 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f646ffff700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 0x7f646403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f646804e0e0 con 0x7f6478109a60 2024-04-26T18:13:46.704 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f64767fc700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 0x7f646403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.904+0000 7f64767fc700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 0x7f646403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:45.914+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f646801f030 con 0x7f6478109a60 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.123+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6458005620 con 0x7f6478109a60 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.126+0000 7f646ffff700 1 -- 172.21.15.37:0/86147296 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6468018e00 con 0x7f6478109a60 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 msgr2=0x7f646403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 0x7f646403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 msgr2=0x7f647819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64680116e0 tx=0x7f6468011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 shutdown_connections 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f64640395b0 0x7f646403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 --2- 172.21.15.37:0/86147296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6478109a60 0x7f647819f080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.128+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 >> 172.21.15.37:0/86147296 conn(0x7f64781034f0 msgr2=0x7f6478103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.129+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 shutdown_connections 2024-04-26T18:13:46.705 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:46.129+0000 7f647d71f700 1 -- 172.21.15.37:0/86147296 wait complete. 2024-04-26T18:13:47.342 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:47 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/86147296' entity='client.admin' 2024-04-26T18:13:48.009 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322f0ca700 1 Processor -- start 2024-04-26T18:13:48.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322f0ca700 1 -- start start 2024-04-26T18:13:48.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322f0ca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:48.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322f0ca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3228107cb0 con 0x7f32281072c0 2024-04-26T18:13:48.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322ce66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:48.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322ce66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43650/0 (socket says 172.21.15.37:43650) 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.116+0000 7f322ce66700 1 -- 172.21.15.37:0/3579343708 learned_addr learned my addr 172.21.15.37:0/3579343708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.117+0000 7f322ce66700 1 -- 172.21.15.37:0/3579343708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322810a250 con 0x7f32281072c0 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.117+0000 7f322ce66700 1 --2- 172.21.15.37:0/3579343708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f321c0098a0 tx=0x7f321c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25a4a9fc24e43a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f32277fe700 1 -- 172.21.15.37:0/3579343708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c004160 con 0x7f32281072c0 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f32277fe700 1 -- 172.21.15.37:0/3579343708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f321c004300 con 0x7f32281072c0 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f32277fe700 1 -- 172.21.15.37:0/3579343708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c011a20 con 0x7f32281072c0 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f322f0ca700 1 -- 172.21.15.37:0/3579343708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 msgr2=0x7f32281076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f322f0ca700 1 --2- 172.21.15.37:0/3579343708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f321c0098a0 tx=0x7f321c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f322f0ca700 1 -- 172.21.15.37:0/3579343708 shutdown_connections 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f322f0ca700 1 --2- 172.21.15.37:0/3579343708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f32281076e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.118+0000 7f322f0ca700 1 -- 172.21.15.37:0/3579343708 >> 172.21.15.37:0/3579343708 conn(0x7f3228102c70 msgr2=0x7f3228105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 -- 172.21.15.37:0/3579343708 shutdown_connections 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 -- 172.21.15.37:0/3579343708 wait complete. 2024-04-26T18:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 Processor -- start 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 -- start start 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.119+0000 7f322f0ca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f322810d600 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.120+0000 7f322ce66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.120+0000 7f322ce66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43652/0 (socket says 172.21.15.37:43652) 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.120+0000 7f322ce66700 1 -- 172.21.15.37:0/946756125 learned_addr learned my addr 172.21.15.37:0/946756125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.120+0000 7f322ce66700 1 -- 172.21.15.37:0/946756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228196500 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.120+0000 7f322ce66700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.121+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c003aa0 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.121+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f321c02c900 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.121+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3228196790 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.121+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f321c034870 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.121+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3228192d50 con 0x7f32281072c0 2024-04-26T18:13:48.017 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.122+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f321c03d3f0 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.122+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228196960 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.122+0000 7f3225ffb700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 0x7f321803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.122+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f321c05eb00 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.122+0000 7f3227fff700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 0x7f321803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.123+0000 7f3227fff700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 0x7f321803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.131+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3228196960 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.384+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3228193470 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.388+0000 7f3225ffb700 1 -- 172.21.15.37:0/946756125 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f321c060e20 con 0x7f32281072c0 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 msgr2=0x7f321803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 0x7f321803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 msgr2=0x7f3228195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:48.018 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 shutdown_connections 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f3218039600 0x7f321803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 --2- 172.21.15.37:0/946756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32281072c0 0x7f3228195fc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 >> 172.21.15.37:0/946756125 conn(0x7f3228102c70 msgr2=0x7f3228103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 shutdown_connections 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:47.391+0000 7f322f0ca700 1 -- 172.21.15.37:0/946756125 wait complete. 2024-04-26T18:13:48.019 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-04-26T18:13:48.672 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:48 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/946756125' entity='client.admin' 2024-04-26T18:13:49.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-mon[29171]: from='mgr.14120 172.21.15.37:0/3175692809' entity='mgr.x' 2024-04-26T18:13:49.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3278381856' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T18:13:49.665 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setuser ceph since I am not root 2024-04-26T18:13:49.665 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setgroup ceph since I am not root 2024-04-26T18:13:49.665 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:49.582+0000 7fc20121a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:13:49.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:49.779+0000 7fc20121a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:49.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:49 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:49.885+0000 7fc20121a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.778+0000 7f1a83f5f700 1 Processor -- start 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.778+0000 7f1a83f5f700 1 -- start start 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.778+0000 7f1a83f5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.778+0000 7f1a83f5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c07ce60 con 0x7f1a7c107870 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a81cfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a81cfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43668/0 (socket says 172.21.15.37:43668) 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a81cfb700 1 -- 172.21.15.37:0/1427402299 learned_addr learned my addr 172.21.15.37:0/1427402299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:50.025 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a81cfb700 1 -- 172.21.15.37:0/1427402299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c1081d0 con 0x7f1a7c107870 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a81cfb700 1 --2- 172.21.15.37:0/1427402299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0098a0 tx=0x7f1a6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b56ffa8cded7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a80cf9700 1 -- 172.21.15.37:0/1427402299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c004160 con 0x7f1a7c107870 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a80cf9700 1 -- 172.21.15.37:0/1427402299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6c004300 con 0x7f1a7c107870 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.779+0000 7f1a80cf9700 1 -- 172.21.15.37:0/1427402299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c011a20 con 0x7f1a7c107870 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 -- 172.21.15.37:0/1427402299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 msgr2=0x7f1a7c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/1427402299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0098a0 tx=0x7f1a6c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 -- 172.21.15.37:0/1427402299 shutdown_connections 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/1427402299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c107c90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 -- 172.21.15.37:0/1427402299 >> 172.21.15.37:0/1427402299 conn(0x7f1a7c103490 msgr2=0x7f1a7c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 -- 172.21.15.37:0/1427402299 shutdown_connections 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.780+0000 7f1a83f5f700 1 -- 172.21.15.37:0/1427402299 wait complete. 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a83f5f700 1 Processor -- start 2024-04-26T18:13:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a83f5f700 1 -- start start 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a83f5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a83f5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c109240 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a81cfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a81cfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43676/0 (socket says 172.21.15.37:43676) 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a81cfb700 1 -- 172.21.15.37:0/3278381856 learned_addr learned my addr 172.21.15.37:0/3278381856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a81cfb700 1 -- 172.21.15.37:0/3278381856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c19f5e0 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.781+0000 7f1a81cfb700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0116e0 tx=0x7f1a6c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.782+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c003b20 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.782+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6c01b8f0 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.782+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c19f870 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.782+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c0238c0 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.782+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a2560 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.783+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1a6c01baa0 con 0x7f1a7c107870 2024-04-26T18:13:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.783+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c064a80 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.783+0000 7f1a72ffd700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 0x7f1a6803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.783+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a6c04e060 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.783+0000 7f1a814fa700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 0x7f1a6803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.784+0000 7f1a814fa700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 0x7f1a6803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a78006fd0 tx=0x7f1a78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:48.788+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a6c01f030 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.061+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1a7c02e720 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.395+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1a6c0292a0 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.397+0000 7f1a72ffd700 1 -- 172.21.15.37:0/3278381856 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1a6c050130 con 0x7f1a7c107870 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 msgr2=0x7f1a6803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 0x7f1a6803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a78006fd0 tx=0x7f1a78006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 msgr2=0x7f1a7c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0116e0 tx=0x7f1a6c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:50.028 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 shutdown_connections 2024-04-26T18:13:50.029 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f1a68039560 0x7f1a6803ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:50.029 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 --2- 172.21.15.37:0/3278381856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c107870 0x7f1a7c19f0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:50.029 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 >> 172.21.15.37:0/3278381856 conn(0x7f1a7c103490 msgr2=0x7f1a7c195d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:50.029 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 shutdown_connections 2024-04-26T18:13:50.029 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:49.399+0000 7f1a83f5f700 1 -- 172.21.15.37:0/3278381856 wait complete. 2024-04-26T18:13:50.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:50 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:50.213+0000 7fc20121a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:13:50.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:50 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:50.306+0000 7fc20121a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:13:50.649 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:50 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3278381856' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T18:13:50.649 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:50 smithi037 ceph-mon[29171]: mgrmap e9: x(active, since 19s) 2024-04-26T18:13:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T18:13:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:13:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba380c2700 1 Processor -- start 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba380c2700 1 -- start start 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba380c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba380c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba301087c0 con 0x7fba30107e60 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba35e5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba35e5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43712/0 (socket says 172.21.15.37:43712) 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.442+0000 7fba35e5e700 1 -- 172.21.15.37:0/1447414486 learned_addr learned my addr 172.21.15.37:0/1447414486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.443+0000 7fba35e5e700 1 -- 172.21.15.37:0/1447414486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba30109050 con 0x7fba30107e60 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.443+0000 7fba35e5e700 1 --2- 172.21.15.37:0/1447414486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0098a0 tx=0x7fba2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b1260a17afe8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.443+0000 7fba34e5c700 1 -- 172.21.15.37:0/1447414486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba2c004160 con 0x7fba30107e60 2024-04-26T18:13:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.443+0000 7fba34e5c700 1 -- 172.21.15.37:0/1447414486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c004300 con 0x7fba30107e60 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.443+0000 7fba34e5c700 1 -- 172.21.15.37:0/1447414486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba2c011a20 con 0x7fba30107e60 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 -- 172.21.15.37:0/1447414486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 msgr2=0x7fba30108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 --2- 172.21.15.37:0/1447414486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0098a0 tx=0x7fba2c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 -- 172.21.15.37:0/1447414486 shutdown_connections 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 --2- 172.21.15.37:0/1447414486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba30108280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 -- 172.21.15.37:0/1447414486 >> 172.21.15.37:0/1447414486 conn(0x7fba301034f0 msgr2=0x7fba30105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 -- 172.21.15.37:0/1447414486 shutdown_connections 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 -- 172.21.15.37:0/1447414486 wait complete. 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.444+0000 7fba380c2700 1 Processor -- start 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba380c2700 1 -- start start 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba380c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:51.318 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba380c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010c310 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba35e5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba35e5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43724/0 (socket says 172.21.15.37:43724) 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba35e5e700 1 -- 172.21.15.37:0/3719720578 learned_addr learned my addr 172.21.15.37:0/3719720578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba35e5e700 1 -- 172.21.15.37:0/3719720578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3019d420 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba35e5e700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba2c000c00 tx=0x7fba2c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba2c003aa0 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2c01b8f0 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3019d6b0 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba2c023890 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.445+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301a03a0 con 0x7fba30107e60 2024-04-26T18:13:51.319 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.446+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fba2c01b440 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.446+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba30064a80 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.447+0000 7fba26ffd700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 0x7fba1c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.447+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fba2c04df20 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.447+0000 7fba3565d700 1 -- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 msgr2=0x7fba1c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.447+0000 7fba3565d700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 0x7fba1c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.452+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba2c01f030 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.648+0000 7fba3565d700 1 -- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 msgr2=0x7fba1c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.648+0000 7fba3565d700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 0x7fba1c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.760+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fba3002e700 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.761+0000 7fba26ffd700 1 -- 172.21.15.37:0/3719720578 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fba2c01bda0 con 0x7fba30107e60 2024-04-26T18:13:51.320 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.765+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 msgr2=0x7fba1c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.765+0000 7fba380c2700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 0x7fba1c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.765+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 msgr2=0x7fba3019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.765+0000 7fba380c2700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba2c000c00 tx=0x7fba2c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.766+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 shutdown_connections 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.766+0000 7fba380c2700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7fba1c039660 0x7fba1c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.766+0000 7fba380c2700 1 --2- 172.21.15.37:0/3719720578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba30107e60 0x7fba3019cee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.766+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 >> 172.21.15.37:0/3719720578 conn(0x7fba301034f0 msgr2=0x7fba30104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.767+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 shutdown_connections 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:50.767+0000 7fba380c2700 1 -- 172.21.15.37:0/3719720578 wait complete. 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-26T18:13:51.321 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-04-26T18:13:51.701 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:51 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3719720578' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:13:51.701 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:51 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:51.568+0000 7fc20121a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:13:51.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:51 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:51.750+0000 7fc20121a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:51.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:51 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:51.927+0000 7fc20121a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:13:52.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:52 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:52.228+0000 7fc20121a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:13:52.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:52 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:52.325+0000 7fc20121a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:13:52.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:52 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:52.536+0000 7fc20121a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:13:52.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:52 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:52.738+0000 7fc20121a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:13:53.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:53.363+0000 7fc20121a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:13:54.119 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:53.805+0000 7fc20121a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:13:54.119 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:53.906+0000 7fc20121a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:13:54.119 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:54 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:54.119+0000 7fc20121a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:13:54.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:54 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:54.687+0000 7fc20121a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:13:54.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:54 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:54.784+0000 7fc20121a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:13:55.402 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:55.042+0000 7fc20121a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:13:55.402 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:55.181+0000 7fc20121a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:13:55.402 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:55.281+0000 7fc20121a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:13:55.403 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:55.402+0000 7fc20121a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:13:55.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:55.715+0000 7fc20121a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:13:56.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:56.345+0000 7fc20121a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:13:56.626 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:13:56.572+0000 7fc20121a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:13:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: Active manager daemon x restarted 2024-04-26T18:13:56.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: Activating manager daemon x 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: mgrmap e10: x(active, starting, since 0.00366927s) 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: Manager daemon x is now available 2024-04-26T18:13:56.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:56] ENGINE Bus STARTING 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:57] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:57] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T18:13:57.947 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: [26/Apr/2024:18:13:57] ENGINE Bus STARTED 2024-04-26T18:13:57.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:57 smithi037 ceph-mon[29171]: mgrmap e11: x(active, since 1.00587s) 2024-04-26T18:13:58.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cb1fb9700 1 Processor -- start 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cb1fb9700 1 -- start start 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cb1fb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cb1fb9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac07ce60 con 0x7f4cac109a40 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43732/0 (socket says 172.21.15.37:43732) 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cab7fe700 1 -- 172.21.15.37:0/3769934481 learned_addr learned my addr 172.21.15.37:0/3769934481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.737+0000 7f4cab7fe700 1 -- 172.21.15.37:0/3769934481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac10c370 con 0x7f4cac109a40 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cab7fe700 1 --2- 172.21.15.37:0/3769934481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0098a0 tx=0x7f4c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18c6df38ede15c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4caa7fc700 1 -- 172.21.15.37:0/3769934481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c004160 con 0x7f4cac109a40 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4caa7fc700 1 -- 172.21.15.37:0/3769934481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c004300 con 0x7f4cac109a40 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4caa7fc700 1 -- 172.21.15.37:0/3769934481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c011a20 con 0x7f4cac109a40 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/3769934481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 msgr2=0x7f4cac10be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/3769934481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0098a0 tx=0x7f4c9c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/3769934481 shutdown_connections 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/3769934481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac10be30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.738+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/3769934481 >> 172.21.15.37:0/3769934481 conn(0x7f4cac103450 msgr2=0x7f4cac105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.739+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/3769934481 shutdown_connections 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.739+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/3769934481 wait complete. 2024-04-26T18:13:58.227 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.739+0000 7f4cb1fb9700 1 Processor -- start 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.739+0000 7f4cb1fb9700 1 -- start start 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.739+0000 7f4cb1fb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cb1fb9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac10d370 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cab7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43742/0 (socket says 172.21.15.37:43742) 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cab7fe700 1 -- 172.21.15.37:0/729237308 learned_addr learned my addr 172.21.15.37:0/729237308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cab7fe700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac19f5a0 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cab7fe700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000c00 tx=0x7f4c9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c003aa0 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c01b8f0 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac19f830 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c9c023870 con 0x7f4cac109a40 2024-04-26T18:13:58.228 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.740+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a2520 con 0x7f4cac109a40 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4c9c023b70 con 0x7f4cac109a40 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4ca8ff9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c9403c1d0 con 0x7f4c94039600 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c9c04ee30 con 0x7f4cac109a40 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4caaffd700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.741+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.942+0000 7f4caaffd700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:51.942+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:52.343+0000 7f4caaffd700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:52.343+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:53.144+0000 7f4caaffd700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:53.144+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:54.745+0000 7f4caaffd700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1233564213 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:54.745+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:56.575+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f4c9c02c910 con 0x7f4cac109a40 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:56.575+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 msgr2=0x7f4c9403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:56.575+0000 7f4ca8ff9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.577+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f4c9c04fd40 con 0x7f4cac109a40 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.577+0000 7f4ca8ff9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 0x7f4c9403f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.577+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c9c02de50 con 0x7f4c9403cfe0 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.577+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 0x7f4c9403f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.578+0000 7f4caaffd700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 0x7f4c9403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0003a10 tx=0x7f4ca00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.579+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4c9c02de50 con 0x7f4c9403cfe0 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.587+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4cac064a80 con 0x7f4c9403cfe0 2024-04-26T18:13:58.230 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4ca8ff9700 1 -- 172.21.15.37:0/729237308 <== mgr.14152 v2:172.21.15.37:6800/813673186 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4cac064a80 con 0x7f4c9403cfe0 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 msgr2=0x7f4c9403f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 0x7f4c9403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0003a10 tx=0x7f4ca00092b0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 msgr2=0x7f4cac19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000c00 tx=0x7f4c9c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 shutdown_connections 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c9403cfe0 0x7f4c9403f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cac109a40 0x7f4cac19f060 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 --2- 172.21.15.37:0/729237308 >> [v2:172.21.15.37:6800/1233564213,v1:172.21.15.37:6801/1233564213] conn(0x7f4c94039600 0x7f4c9403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 >> 172.21.15.37:0/729237308 conn(0x7f4cac103450 msgr2=0x7f4cac103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 shutdown_connections 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:57.588+0000 7f4cb1fb9700 1 -- 172.21.15.37:0/729237308 wait complete. 2024-04-26T18:13:58.231 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-04-26T18:13:58.232 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-04-26T18:13:58.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:58 smithi037 ceph-mon[29171]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:13:58.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:58 smithi037 ceph-mon[29171]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:13:59.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T18:13:59.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.862+0000 7f0c72d70700 1 Processor -- start 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c72d70700 1 -- start start 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c72d70700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c72d70700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c109960 con 0x7f0c6c108f70 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c70b0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c70b0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36722/0 (socket says 172.21.15.37:36722) 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c70b0c700 1 -- 172.21.15.37:0/2115769736 learned_addr learned my addr 172.21.15.37:0/2115769736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c70b0c700 1 -- 172.21.15.37:0/2115769736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c10a170 con 0x7f0c6c108f70 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.863+0000 7f0c70b0c700 1 --2- 172.21.15.37:0/2115769736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098a0 tx=0x7f0c5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90f2cf2826684bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c6b7fe700 1 -- 172.21.15.37:0/2115769736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c004160 con 0x7f0c6c108f70 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c6b7fe700 1 -- 172.21.15.37:0/2115769736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c5c004300 con 0x7f0c6c108f70 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c6b7fe700 1 -- 172.21.15.37:0/2115769736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c011a20 con 0x7f0c6c108f70 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 -- 172.21.15.37:0/2115769736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 msgr2=0x7f0c6c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:59.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 --2- 172.21.15.37:0/2115769736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0098a0 tx=0x7f0c5c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 -- 172.21.15.37:0/2115769736 shutdown_connections 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 --2- 172.21.15.37:0/2115769736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c109390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 -- 172.21.15.37:0/2115769736 >> 172.21.15.37:0/2115769736 conn(0x7f0c6c1044f0 msgr2=0x7f0c6c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 -- 172.21.15.37:0/2115769736 shutdown_connections 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.864+0000 7f0c72d70700 1 -- 172.21.15.37:0/2115769736 wait complete. 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c72d70700 1 Processor -- start 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c72d70700 1 -- start start 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c72d70700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c72d70700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10d520 con 0x7f0c6c108f70 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c70b0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c70b0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36730/0 (socket says 172.21.15.37:36730) 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c70b0c700 1 -- 172.21.15.37:0/112509192 learned_addr learned my addr 172.21.15.37:0/112509192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:13:59.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.865+0000 7f0c70b0c700 1 -- 172.21.15.37:0/112509192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c07f0a0 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c70b0c700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009520 tx=0x7f0c5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c0039e0 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c5c01b8f0 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c07d290 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c019770 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.866+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c07d7d0 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0c5c019a20 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c69ffb700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 0x7f0c5403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c5c04dfc0 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c6bfff700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 0x7f0c5403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c064a80 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.867+0000 7f0c6bfff700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 0x7f0c5403bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c60006fd0 tx=0x7f0c60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:58.873+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c5c01f030 con 0x7f0c6c108f70 2024-04-26T18:13:59.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.091+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0c6c07e140 con 0x7f0c54039650 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.216+0000 7f0c69ffb700 1 -- 172.21.15.37:0/112509192 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0c6000a040 con 0x7f0c54039650 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 msgr2=0x7f0c5403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 0x7f0c5403bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c60006fd0 tx=0x7f0c60006e40 comp rx=0 tx=0).stop 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 msgr2=0x7f0c6c07eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009520 tx=0x7f0c5c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 shutdown_connections 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0c54039650 0x7f0c5403bb10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 --2- 172.21.15.37:0/112509192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c6c108f70 0x7f0c6c07eb60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 >> 172.21.15.37:0/112509192 conn(0x7f0c6c1044f0 msgr2=0x7f0c6c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 shutdown_connections 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:13:59.219+0000 7f0c72d70700 1 -- 172.21.15.37:0/112509192 wait complete. 2024-04-26T18:13:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-04-26T18:13:59.935 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:59 smithi037 ceph-mon[29171]: mgrmap e12: x(active, since 2s) 2024-04-26T18:13:59.936 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:13:59.936 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:59 smithi037 ceph-mon[29171]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:13:59.936 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:13:59.936 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:13:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:01.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:01.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:02.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$utAFn.gmgBkDrmTuloWAXe3eZs1ROxhhoWd5O7FBVwJIhgDC1S23a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714155240, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T18:14:02.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.317+0000 7fc398a97700 1 Processor -- start 2024-04-26T18:14:02.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.317+0000 7fc398a97700 1 -- start start 2024-04-26T18:14:02.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc398a97700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:02.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc398a97700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc394076ce0 con 0x7fc3941078b0 2024-04-26T18:14:02.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc39259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:02.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc39259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36732/0 (socket says 172.21.15.37:36732) 2024-04-26T18:14:02.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc39259c700 1 -- 172.21.15.37:0/210470612 learned_addr learned my addr 172.21.15.37:0/210470612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:02.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc39259c700 1 -- 172.21.15.37:0/210470612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc394108210 con 0x7fc3941078b0 2024-04-26T18:14:02.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.318+0000 7fc39259c700 1 --2- 172.21.15.37:0/210470612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61b286e898d036d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:02.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc39159a700 1 -- 172.21.15.37:0/210470612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388004160 con 0x7fc3941078b0 2024-04-26T18:14:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc39159a700 1 -- 172.21.15.37:0/210470612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc388004300 con 0x7fc3941078b0 2024-04-26T18:14:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc39159a700 1 -- 172.21.15.37:0/210470612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388011a20 con 0x7fc3941078b0 2024-04-26T18:14:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc398a97700 1 -- 172.21.15.37:0/210470612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 msgr2=0x7fc394107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc398a97700 1 --2- 172.21.15.37:0/210470612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).stop 2024-04-26T18:14:02.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc398a97700 1 -- 172.21.15.37:0/210470612 shutdown_connections 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc398a97700 1 --2- 172.21.15.37:0/210470612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc394107cd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.319+0000 7fc398a97700 1 -- 172.21.15.37:0/210470612 >> 172.21.15.37:0/210470612 conn(0x7fc3941034f0 msgr2=0x7fc394105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.320+0000 7fc398a97700 1 -- 172.21.15.37:0/210470612 shutdown_connections 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.320+0000 7fc398a97700 1 -- 172.21.15.37:0/210470612 wait complete. 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc398a97700 1 Processor -- start 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc398a97700 1 -- start start 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc398a97700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc398a97700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc394109280 con 0x7fc3941078b0 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc39259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:02.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc39259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36738/0 (socket says 172.21.15.37:36738) 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc39259c700 1 -- 172.21.15.37:0/1701661604 learned_addr learned my addr 172.21.15.37:0/1701661604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc39259c700 1 -- 172.21.15.37:0/1701661604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39419f5d0 con 0x7fc3941078b0 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.321+0000 7fc39259c700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc388004660 tx=0x7fc388011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.322+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388003aa0 con 0x7fc3941078b0 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.322+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38801b8f0 con 0x7fc3941078b0 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.322+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39419f860 con 0x7fc3941078b0 2024-04-26T18:14:02.137 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.322+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3941a2550 con 0x7fc3941078b0 2024-04-26T18:14:02.138 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.322+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388023900 con 0x7fc3941078b0 2024-04-26T18:14:02.138 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc388023bc0 con 0x7fc3941078b0 2024-04-26T18:14:02.138 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc3837fe700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 0x7fc37c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:02.138 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc394064a80 con 0x7fc3941078b0 2024-04-26T18:14:02.138 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc38804e3c0 con 0x7fc3941078b0 2024-04-26T18:14:02.139 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc391d9b700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 0x7fc37c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:02.139 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.323+0000 7fc391d9b700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 0x7fc37c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:02.139 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.328+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc38801f030 con 0x7fc3941078b0 2024-04-26T18:14:02.139 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.553+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc39419fd40 con 0x7fc37c039600 2024-04-26T18:14:02.139 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.800+0000 7fc3837fe700 1 -- 172.21.15.37:0/1701661604 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc39419fd40 con 0x7fc37c039600 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 msgr2=0x7fc37c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 0x7fc37c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 msgr2=0x7fc39419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc388004660 tx=0x7fc388011aa0 comp rx=0 tx=0).stop 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 shutdown_connections 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc37c039600 0x7fc37c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 --2- 172.21.15.37:0/1701661604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3941078b0 0x7fc39419f090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 >> 172.21.15.37:0/1701661604 conn(0x7fc3941034f0 msgr2=0x7fc394195cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.802+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 shutdown_connections 2024-04-26T18:14:02.140 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:00.803+0000 7fc398a97700 1 -- 172.21.15.37:0/1701661604 wait complete. 2024-04-26T18:14:02.141 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-04-26T18:14:02.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:02 smithi037 ceph-mon[29171]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:03.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:03 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1304392588' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T18:14:03.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.527+0000 7f8c0f71b700 1 Processor -- start 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.527+0000 7f8c0f71b700 1 -- start start 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.527+0000 7f8c0f71b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.527+0000 7f8c0f71b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c081099d0 con 0x7f8c08108fe0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.527+0000 7f8c0d4b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8c0d4b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36748/0 (socket says 172.21.15.37:36748) 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8c0d4b7700 1 -- 172.21.15.37:0/1247560113 learned_addr learned my addr 172.21.15.37:0/1247560113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8c0d4b7700 1 -- 172.21.15.37:0/1247560113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0810a1e0 con 0x7f8c08108fe0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8c0d4b7700 1 --2- 172.21.15.37:0/1247560113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39eb824c41c18e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8bfffff700 1 -- 172.21.15.37:0/1247560113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04004160 con 0x7f8c08108fe0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8bfffff700 1 -- 172.21.15.37:0/1247560113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c04004300 con 0x7f8c08108fe0 2024-04-26T18:14:03.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.528+0000 7f8bfffff700 1 -- 172.21.15.37:0/1247560113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04011a20 con 0x7f8c08108fe0 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1247560113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 msgr2=0x7f8c08109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1247560113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c04009bb0 comp rx=0 tx=0).stop 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1247560113 shutdown_connections 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1247560113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c08109400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1247560113 >> 172.21.15.37:0/1247560113 conn(0x7f8c081045a0 msgr2=0x7f8c081069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1247560113 shutdown_connections 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1247560113 wait complete. 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.529+0000 7f8c0f71b700 1 Processor -- start 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0f71b700 1 -- start start 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0f71b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0f71b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0810d590 con 0x7f8c08108fe0 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0d4b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:03.294 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0d4b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36754/0 (socket says 172.21.15.37:36754) 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0d4b7700 1 -- 172.21.15.37:0/1304392588 learned_addr learned my addr 172.21.15.37:0/1304392588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0d4b7700 1 -- 172.21.15.37:0/1304392588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0807f0a0 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0d4b7700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c04000c00 tx=0x7f8c04011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04003aa0 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0401b8f0 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0807d290 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.530+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c04023890 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.531+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0807d7d0 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.531+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8c0401b440 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.531+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c08064a80 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.532+0000 7f8bfe7fc700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 0x7f8bf403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.532+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8c0404df80 con 0x7f8c08108fe0 2024-04-26T18:14:03.295 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.532+0000 7f8c0ccb6700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 0x7f8bf403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.532+0000 7f8c0ccb6700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 0x7f8bf403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.537+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c0401f030 con 0x7f8c08108fe0 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.745+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f8c0807da50 con 0x7f8c08108fe0 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.745+0000 7f8bfe7fc700 1 -- 172.21.15.37:0/1304392588 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8c040292a0 con 0x7f8c08108fe0 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 msgr2=0x7f8bf403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 0x7f8bf403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 msgr2=0x7f8c0807eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c04000c00 tx=0x7f8c04011aa0 comp rx=0 tx=0).stop 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 shutdown_connections 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8bf4039600 0x7f8bf403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 --2- 172.21.15.37:0/1304392588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c08108fe0 0x7f8c0807eb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.747+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 >> 172.21.15.37:0/1304392588 conn(0x7f8c081045a0 msgr2=0x7f8c08105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.748+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 shutdown_connections 2024-04-26T18:14:03.296 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:02.748+0000 7f8c0f71b700 1 -- 172.21.15.37:0/1304392588 wait complete. 2024-04-26T18:14:03.300 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:14:03.300 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T18:14:03.308 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:14:03.308 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T18:14:03.308 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T18:14:03.308 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T18:14:03.309 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037:8443/ 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: Password: 92f3sovuxz 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:03.310 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config directory 2024-04-26T18:14:03.311 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-04-26T18:14:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:04 smithi037 ceph-mon[29171]: mgrmap e13: x(active, since 6s) 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c50c23700 1 Processor -- start 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c50c23700 1 -- start start 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c50c23700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c50c23700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c109960 con 0x7f4c4c108f70 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36762/0 (socket says 172.21.15.37:36762) 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.698+0000 7f4c4a59c700 1 -- 172.21.15.37:0/3935863189 learned_addr learned my addr 172.21.15.37:0/3935863189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c4a59c700 1 -- 172.21.15.37:0/3935863189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c10a170 con 0x7f4c4c108f70 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c4a59c700 1 --2- 172.21.15.37:0/3935863189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098a0 tx=0x7f4c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49b7a3a4481118 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c4959a700 1 -- 172.21.15.37:0/3935863189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c004160 con 0x7f4c4c108f70 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c4959a700 1 -- 172.21.15.37:0/3935863189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3c004300 con 0x7f4c4c108f70 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c4959a700 1 -- 172.21.15.37:0/3935863189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c011a20 con 0x7f4c4c108f70 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c50c23700 1 -- 172.21.15.37:0/3935863189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 msgr2=0x7f4c4c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.699+0000 7f4c50c23700 1 --2- 172.21.15.37:0/3935863189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098a0 tx=0x7f4c3c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 -- 172.21.15.37:0/3935863189 shutdown_connections 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 --2- 172.21.15.37:0/3935863189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c109390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 -- 172.21.15.37:0/3935863189 >> 172.21.15.37:0/3935863189 conn(0x7f4c4c1044f0 msgr2=0x7f4c4c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 -- 172.21.15.37:0/3935863189 shutdown_connections 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 -- 172.21.15.37:0/3935863189 wait complete. 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 Processor -- start 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.700+0000 7f4c50c23700 1 -- start start 2024-04-26T18:14:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c50c23700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c50c23700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10d520 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36766/0 (socket says 172.21.15.37:36766) 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c4a59c700 1 -- 172.21.15.37:0/4284387466 learned_addr learned my addr 172.21.15.37:0/4284387466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c4a59c700 1 -- 172.21.15.37:0/4284387466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c07f0a0 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c4a59c700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009520 tx=0x7f4c3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c0039e0 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3c01b8f0 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c07d290 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c019770 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.701+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c07d7d0 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.702+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4c3c01b440 con 0x7f4c4c108f70 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.702+0000 7f4c3b7fe700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 0x7f4c3403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:04.510 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.702+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c064a80 con 0x7f4c4c108f70 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.703+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c3c04e170 con 0x7f4c4c108f70 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.703+0000 7f4c49d9b700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 0x7f4c3403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.703+0000 7f4c49d9b700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 0x7f4c3403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006fd0 tx=0x7f4c40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.708+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c3c01f030 con 0x7f4c4c108f70 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.912+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4c4c07e0c0 con 0x7f4c4c108f70 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.913+0000 7f4c3b7fe700 1 -- 172.21.15.37:0/4284387466 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4c4c07e0c0 con 0x7f4c4c108f70 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 msgr2=0x7f4c3403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 0x7f4c3403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006fd0 tx=0x7f4c40006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 msgr2=0x7f4c4c07eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009520 tx=0x7f4c3c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 shutdown_connections 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4c340396a0 0x7f4c3403bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:04.511 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 --2- 172.21.15.37:0/4284387466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c108f70 0x7f4c4c07eb60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:04.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 >> 172.21.15.37:0/4284387466 conn(0x7f4c4c1044f0 msgr2=0x7f4c4c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:04.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 shutdown_connections 2024-04-26T18:14:04.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:03.915+0000 7f4c50c23700 1 -- 172.21.15.37:0/4284387466 wait complete. 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667f93b700 1 Processor -- start 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667f93b700 1 -- start start 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667f93b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667f93b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6678107c30 con 0x7f6678107240 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667d6d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:05.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.948+0000 7f667d6d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54404/0 (socket says 172.21.15.37:54404) 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f667d6d7700 1 -- 172.21.15.37:0/2714126426 learned_addr learned my addr 172.21.15.37:0/2714126426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f667d6d7700 1 -- 172.21.15.37:0/2714126426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667810a1d0 con 0x7f6678107240 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f667d6d7700 1 --2- 172.21.15.37:0/2714126426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66740098a0 tx=0x7f6674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626e73cef932140c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f666ffff700 1 -- 172.21.15.37:0/2714126426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6674004160 con 0x7f6678107240 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f666ffff700 1 -- 172.21.15.37:0/2714126426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6674004300 con 0x7f6678107240 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.949+0000 7f666ffff700 1 -- 172.21.15.37:0/2714126426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6674011a20 con 0x7f6678107240 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 -- 172.21.15.37:0/2714126426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 msgr2=0x7f6678107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 --2- 172.21.15.37:0/2714126426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66740098a0 tx=0x7f6674009bb0 comp rx=0 tx=0).stop 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 -- 172.21.15.37:0/2714126426 shutdown_connections 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 --2- 172.21.15.37:0/2714126426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f6678107660 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 -- 172.21.15.37:0/2714126426 >> 172.21.15.37:0/2714126426 conn(0x7f6678102b70 msgr2=0x7f6678104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:05.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 -- 172.21.15.37:0/2714126426 shutdown_connections 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.950+0000 7f667f93b700 1 -- 172.21.15.37:0/2714126426 wait complete. 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667f93b700 1 Processor -- start 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667f93b700 1 -- start start 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667f93b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667f93b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f667810d580 con 0x7f6678107240 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667d6d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667d6d7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54412/0 (socket says 172.21.15.37:54412) 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667d6d7700 1 -- 172.21.15.37:0/1231287065 learned_addr learned my addr 172.21.15.37:0/1231287065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667d6d7700 1 -- 172.21.15.37:0/1231287065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667819f600 con 0x7f6678107240 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.951+0000 7f667d6d7700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66740119f0 tx=0x7f6674011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:05.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66740039e0 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667401b8f0 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667819f890 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66740237a0 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781a2580 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.952+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f667402c3f0 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.953+0000 7f666e7fc700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 0x7f666403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.953+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f667404e0c0 con 0x7f6678107240 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.953+0000 7f667ced6700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 0x7f666403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:05.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.953+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6678064a80 con 0x7f6678107240 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.953+0000 7f667ced6700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 0x7f666403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:04.958+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f667401f030 con 0x7f6678107240 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.281+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f66781a2970 con 0x7f6678107240 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.284+0000 7f666e7fc700 1 -- 172.21.15.37:0/1231287065 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f66740105e0 con 0x7f6678107240 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 msgr2=0x7f666403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 0x7f666403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 msgr2=0x7f667819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66740119f0 tx=0x7f6674011ad0 comp rx=0 tx=0).stop 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 shutdown_connections 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f66640396b0 0x7f666403bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 --2- 172.21.15.37:0/1231287065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6678107240 0x7f667819f0c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:05.829 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 >> 172.21.15.37:0/1231287065 conn(0x7f6678102b70 msgr2=0x7f6678103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 shutdown_connections 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T18:14:05.286+0000 7f667f93b700 1 -- 172.21.15.37:0/1231287065 wait complete. 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.830 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.831 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T18:14:05.832 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:05.833 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-04-26T18:14:05.865 INFO:tasks.cephadm:Fetching config... 2024-04-26T18:14:05.866 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:05.866 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T18:14:05.892 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T18:14:05.892 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:05.892 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T18:14:05.952 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T18:14:05.952 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:05.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T18:14:06.019 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T18:14:06.019 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:06.019 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T18:14:06.076 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T18:14:06.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:14:06.155 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:06.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:14:06.213 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:06.225 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:14:06.264 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzt5ACNZNqjc9jl/2Odl0k/G1wNwXeI6yRbyqSlQqAZF5PG9gL+WzWC6o6SvXedO0LRvtMrLAQyqC+Gb4DEQ/Va+L1KYujaIhAC6t6mIaaEbwa1+ARgyyFgoMQvEGiaesEFX8scUJz60+dSDf122GsUa3kx2Nni5FppOV4BJW/rmu46lBnB7p6b4XLPOowMJC3L6bZ6Gu4C2Y3U0Ltx9SPeAPItJ6K/lV3rM1NnR5V15pJB1i9Zzqh5+DcWw6cJutR3ik+VEamFCZMqWR+YN+jLlTRPViBI+mDPEPsuwZUtQnCQlQfovSYA+znkOxkALSzAUjaR8O0QPcfNwvF19zzV6PzTFZWKy4aafLKXYbb5Nb0bWjbRbchUMjwJYTBBCafUdjUirc+6bpEAmu24VT1aygrVgZwHlfnKOsr+naclLrKV/Wa7fXoEn/sqAYA4TViQZodtWQMkJHXvcs1JNu1mlZybP21AN5kp0xz8RPCM5jKwYgs8ll/nX1GeaogaqM= ceph-755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:06.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T18:14:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:06.635 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:06 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1231287065' entity='client.admin' 2024-04-26T18:14:08.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 -- 172.21.15.37:0/1056627419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 msgr2=0x7fde80104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:08.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 --2- 172.21.15.37:0/1056627419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde80104900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde70009910 tx=0x7fde70009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:08.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 -- 172.21.15.37:0/1056627419 shutdown_connections 2024-04-26T18:14:08.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 --2- 172.21.15.37:0/1056627419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde80104900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 -- 172.21.15.37:0/1056627419 >> 172.21.15.37:0/1056627419 conn(0x7fde800fe470 msgr2=0x7fde80100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 -- 172.21.15.37:0/1056627419 shutdown_connections 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.788+0000 7fde853b6700 1 -- 172.21.15.37:0/1056627419 wait complete. 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde853b6700 1 Processor -- start 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde853b6700 1 -- start start 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde853b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde853b6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde80108e50 con 0x7fde801044e0 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde7effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde7effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54432/0 (socket says 172.21.15.37:54432) 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.789+0000 7fde7effd700 1 -- 172.21.15.37:0/1150706094 learned_addr learned my addr 172.21.15.37:0/1150706094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde7effd700 1 -- 172.21.15.37:0/1150706094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde80198b20 con 0x7fde801044e0 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde7effd700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde70005800 tx=0x7fde70009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:08.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde70027070 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7000be10 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde80198db0 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde7000f460 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.790+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8019baa0 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.791+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fde7002a070 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.791+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde801095d0 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.791+0000 7fde67fff700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 0x7fde6803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.791+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde7001e030 con 0x7fde801044e0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.791+0000 7fde7e7fc700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 0x7fde6803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:08.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.792+0000 7fde7e7fc700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 0x7fde6803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:08.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:08.796+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde70054080 con 0x7fde801044e0 2024-04-26T18:14:09.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.005+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fde8002e490 con 0x7fde801044e0 2024-04-26T18:14:09.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.007+0000 7fde67fff700 1 -- 172.21.15.37:0/1150706094 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fde70031020 con 0x7fde801044e0 2024-04-26T18:14:09.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 msgr2=0x7fde6803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 0x7fde6803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fde74006fd0 tx=0x7fde74006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 msgr2=0x7fde801985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde70005800 tx=0x7fde70009f60 comp rx=0 tx=0).stop 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 shutdown_connections 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fde68039650 0x7fde6803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 --2- 172.21.15.37:0/1150706094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde801044e0 0x7fde801985e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.009+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 >> 172.21.15.37:0/1150706094 conn(0x7fde800fe470 msgr2=0x7fde800fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.010+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 shutdown_connections 2024-04-26T18:14:09.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:09.010+0000 7fde853b6700 1 -- 172.21.15.37:0/1150706094 wait complete. 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:09.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:09.573 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T18:14:09.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T18:14:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:10.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:10 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1150706094' entity='client.admin' 2024-04-26T18:14:10.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:10.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:11.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.271+0000 7f342bbb6700 1 -- 172.21.15.37:0/2254127474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 msgr2=0x7f3424103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.271+0000 7f342bbb6700 1 --2- 172.21.15.37:0/2254127474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f3424103290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3414009910 tx=0x7f3414009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.271+0000 7f342bbb6700 1 -- 172.21.15.37:0/2254127474 shutdown_connections 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.271+0000 7f342bbb6700 1 --2- 172.21.15.37:0/2254127474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f3424103290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 -- 172.21.15.37:0/2254127474 >> 172.21.15.37:0/2254127474 conn(0x7f34240fe420 msgr2=0x7f34241008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 -- 172.21.15.37:0/2254127474 shutdown_connections 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 -- 172.21.15.37:0/2254127474 wait complete. 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 Processor -- start 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 -- start start 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.272+0000 7f342bbb6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3424108d70 con 0x7f3424102e70 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f3429952700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f3429952700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54454/0 (socket says 172.21.15.37:54454) 2024-04-26T18:14:11.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f3429952700 1 -- 172.21.15.37:0/1653946667 learned_addr learned my addr 172.21.15.37:0/1653946667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f3429952700 1 -- 172.21.15.37:0/1653946667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3424077620 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f3429952700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3414005950 tx=0x7f3414004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3414026070 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341400bc00 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3424075810 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3414003a10 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.273+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3424075d50 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.274+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3414029070 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.274+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34241094f0 con 0x7f3424102e70 2024-04-26T18:14:11.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.274+0000 7f341affd700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 0x7f341003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:11.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.275+0000 7f3429151700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 0x7f341003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:11.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.275+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f341401e070 con 0x7f3424102e70 2024-04-26T18:14:11.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.275+0000 7f3429151700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 0x7f341003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:11.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.280+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3414013070 con 0x7f3424102e70 2024-04-26T18:14:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.493+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f34241917f0 con 0x7f3410039650 2024-04-26T18:14:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.497+0000 7f341affd700 1 -- 172.21.15.37:0/1653946667 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34241917f0 con 0x7f3410039650 2024-04-26T18:14:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 msgr2=0x7f341003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 0x7f341003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 msgr2=0x7f34240770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3414005950 tx=0x7f3414004fa0 comp rx=0 tx=0).stop 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 shutdown_connections 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3410039650 0x7f341003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 --2- 172.21.15.37:0/1653946667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424102e70 0x7f34240770e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 >> 172.21.15.37:0/1653946667 conn(0x7f34240fe420 msgr2=0x7f34240fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 shutdown_connections 2024-04-26T18:14:11.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:11.499+0000 7f342bbb6700 1 -- 172.21.15.37:0/1653946667 wait complete. 2024-04-26T18:14:12.273 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-04-26T18:14:12.273 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:14:12.274 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:14:12.293 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:14:12.294 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:12.352 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-04-26T18:14:12.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch host add smithi120 2024-04-26T18:14:12.596 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:12.807 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:12.807 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:12.808 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:12.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:14.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 -- 172.21.15.37:0/1688105896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 msgr2=0x7fef981023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 --2- 172.21.15.37:0/1688105896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef981023b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fef9000b0a0 tx=0x7fef9000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 -- 172.21.15.37:0/1688105896 shutdown_connections 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 --2- 172.21.15.37:0/1688105896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef981023b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 -- 172.21.15.37:0/1688105896 >> 172.21.15.37:0/1688105896 conn(0x7fef98077880 msgr2=0x7fef98077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 -- 172.21.15.37:0/1688105896 shutdown_connections 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.053+0000 7fef9ed51700 1 -- 172.21.15.37:0/1688105896 wait complete. 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9ed51700 1 Processor -- start 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9ed51700 1 -- start start 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9ed51700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:14.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9ed51700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810c200 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9caed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9caed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54472/0 (socket says 172.21.15.37:54472) 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9caed700 1 -- 172.21.15.37:0/4107670902 learned_addr learned my addr 172.21.15.37:0/4107670902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.054+0000 7fef9caed700 1 -- 172.21.15.37:0/4107670902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef981058c0 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef9caed700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef90006130 tx=0x7fef900097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef90021020 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef900079b0 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef90017040 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef98103f60 con 0x7fef98101fd0 2024-04-26T18:14:14.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.055+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef98104420 con 0x7fef98101fd0 2024-04-26T18:14:14.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.056+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fef90024020 con 0x7fef98101fd0 2024-04-26T18:14:14.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.056+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98103460 con 0x7fef98101fd0 2024-04-26T18:14:14.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.056+0000 7fef95ffb700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 0x7fef8003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:14.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.056+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef9000f050 con 0x7fef98101fd0 2024-04-26T18:14:14.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.056+0000 7fef97fff700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 0x7fef8003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:14.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.057+0000 7fef97fff700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 0x7fef8003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef8c006fd0 tx=0x7fef8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:14.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.061+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef9002b020 con 0x7fef98101fd0 2024-04-26T18:14:14.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:14.271+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}) -- 0x7fef98104f70 con 0x7fef80039650 2024-04-26T18:14:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:14 smithi037 ceph-mon[29171]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:15.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:15 smithi037 ceph-mon[29171]: Deploying cephadm binary to smithi120 2024-04-26T18:14:17.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:17 smithi037 ceph-mon[29171]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:18.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.646+0000 7fef95ffb700 1 -- 172.21.15.37:0/4107670902 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fef98104f70 con 0x7fef80039650 2024-04-26T18:14:18.646 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-04-26T18:14:18.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 msgr2=0x7fef8003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:18.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 0x7fef8003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef8c006fd0 tx=0x7fef8c006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:18.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 msgr2=0x7fef98103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef90006130 tx=0x7fef900097d0 comp rx=0 tx=0).stop 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 shutdown_connections 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fef80039650 0x7fef8003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 --2- 172.21.15.37:0/4107670902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef98101fd0 0x7fef98103a20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 >> 172.21.15.37:0/4107670902 conn(0x7fef98077880 msgr2=0x7fef9806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 shutdown_connections 2024-04-26T18:14:18.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:18.649+0000 7fef9ed51700 1 -- 172.21.15.37:0/4107670902 wait complete. 2024-04-26T18:14:19.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:14:19.499 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:19.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:19 smithi037 ceph-mon[29171]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:19.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:19 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:19.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:19 smithi037 ceph-mon[29171]: Added host smithi120 2024-04-26T18:14:19.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:19 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:19.932 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:19 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:20.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 -- 172.21.15.37:0/2294128433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 msgr2=0x7f4088075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 --2- 172.21.15.37:0/2294128433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f4088075b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f407800b0a0 tx=0x7f407800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 -- 172.21.15.37:0/2294128433 shutdown_connections 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 --2- 172.21.15.37:0/2294128433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f4088075b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 -- 172.21.15.37:0/2294128433 >> 172.21.15.37:0/2294128433 conn(0x7f40880fe940 msgr2=0x7f4088100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 -- 172.21.15.37:0/2294128433 shutdown_connections 2024-04-26T18:14:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.029+0000 7f408cdae700 1 -- 172.21.15.37:0/2294128433 wait complete. 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.030+0000 7f408cdae700 1 Processor -- start 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.030+0000 7f408cdae700 1 -- start start 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.030+0000 7f408cdae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.030+0000 7f408cdae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f408810f570 con 0x7f4088075740 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.030+0000 7f408659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:21.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43218/0 (socket says 172.21.15.37:43218) 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408659c700 1 -- 172.21.15.37:0/515653281 learned_addr learned my addr 172.21.15.37:0/515653281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408659c700 1 -- 172.21.15.37:0/515653281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40881a0be0 con 0x7f4088075740 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408659c700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4078015040 tx=0x7f4078009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4078021020 con 0x7f4088075740 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4078004000 con 0x7f4088075740 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4078017040 con 0x7f4088075740 2024-04-26T18:14:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40881a0e70 con 0x7f4088075740 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.031+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408819a860 con 0x7f4088075740 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.032+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4078024020 con 0x7f4088075740 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.032+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4088076e80 con 0x7f4088075740 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.032+0000 7f407f7fe700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 0x7f407403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.032+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f407800f050 con 0x7f4088075740 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.033+0000 7f4085d9b700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 0x7f407403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.033+0000 7f4085d9b700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 0x7f407403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4070006fd0 tx=0x7f4070006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:21.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.038+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f407802b020 con 0x7f4088075740 2024-04-26T18:14:21.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.252+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f408819bd90 con 0x7f4074039650 2024-04-26T18:14:21.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.253+0000 7f407f7fe700 1 -- 172.21.15.37:0/515653281 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f408819bd90 con 0x7f4074039650 2024-04-26T18:14:21.253 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:21.253 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 msgr2=0x7f407403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 0x7f407403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4070006fd0 tx=0x7f4070006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 msgr2=0x7f40881a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4078015040 tx=0x7f4078009800 comp rx=0 tx=0).stop 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 shutdown_connections 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4074039650 0x7f407403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 --2- 172.21.15.37:0/515653281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4088075740 0x7f40881a06a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:21.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 >> 172.21.15.37:0/515653281 conn(0x7f40880fe940 msgr2=0x7f4088107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:21.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 shutdown_connections 2024-04-26T18:14:21.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:21.255+0000 7f408cdae700 1 -- 172.21.15.37:0/515653281 wait complete. 2024-04-26T18:14:21.905 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-04-26T18:14:21.905 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:14:21.906 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:14:21.925 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:14:21.925 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:21.982 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-04-26T18:14:21.982 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch host add smithi170 2024-04-26T18:14:22.228 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:22.519 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:22 smithi037 ceph-mon[29171]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:22.519 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:22 smithi037 ceph-mon[29171]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:22.519 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/1531311823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 msgr2=0x7f9adc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/1531311823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc1043d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009910 tx=0x7f9acc009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/1531311823 shutdown_connections 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/1531311823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc1043d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/1531311823 >> 172.21.15.37:0/1531311823 conn(0x7f9adc0fd840 msgr2=0x7f9adc0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/1531311823 shutdown_connections 2024-04-26T18:14:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/1531311823 wait complete. 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.770+0000 7f9ae0b4f700 1 Processor -- start 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ae0b4f700 1 -- start start 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ae0b4f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ae0b4f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc10d390 con 0x7f9adc103ff0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ada59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ada59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43246/0 (socket says 172.21.15.37:43246) 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ada59c700 1 -- 172.21.15.37:0/4005801666 learned_addr learned my addr 172.21.15.37:0/4005801666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ada59c700 1 -- 172.21.15.37:0/4005801666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc103b40 con 0x7f9adc103ff0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.771+0000 7f9ada59c700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0056d0 tx=0x7f9acc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9acc026070 con 0x7f9adc103ff0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9acc004370 con 0x7f9adc103ff0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc102190 con 0x7f9adc103ff0 2024-04-26T18:14:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9acc0039b0 con 0x7f9adc103ff0 2024-04-26T18:14:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc102650 con 0x7f9adc103ff0 2024-04-26T18:14:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.772+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9acc029070 con 0x7f9adc103ff0 2024-04-26T18:14:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.773+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc06bbb0 con 0x7f9adc103ff0 2024-04-26T18:14:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.773+0000 7f9ad37fe700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 0x7f9ac803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.773+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9acc01e070 con 0x7f9adc103ff0 2024-04-26T18:14:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.773+0000 7f9ad3fff700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 0x7f9ac803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.773+0000 7f9ad3fff700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 0x7f9ac803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006fd0 tx=0x7f9ac4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.778+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9acc013070 con 0x7f9adc103ff0 2024-04-26T18:14:23.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:23.992+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7f9adc103180 con 0x7f9ac8039650 2024-04-26T18:14:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:24 smithi037 ceph-mon[29171]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:25.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:25.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:25 smithi037 ceph-mon[29171]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:26.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:26 smithi037 ceph-mon[29171]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:26.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:26 smithi037 ceph-mon[29171]: Deploying cephadm binary to smithi170 2024-04-26T18:14:26.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:26 smithi037 ceph-mon[29171]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:26 smithi037 ceph-mon[29171]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:28.249 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-04-26T18:14:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.249+0000 7f9ad37fe700 1 -- 172.21.15.37:0/4005801666 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9adc103180 con 0x7f9ac8039650 2024-04-26T18:14:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 msgr2=0x7f9ac803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 0x7f9ac803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4006fd0 tx=0x7f9ac4006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 msgr2=0x7f9adc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0056d0 tx=0x7f9acc0057b0 comp rx=0 tx=0).stop 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 shutdown_connections 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9ac8039650 0x7f9ac803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 --2- 172.21.15.37:0/4005801666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9adc103ff0 0x7f9adc101c50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 >> 172.21.15.37:0/4005801666 conn(0x7f9adc0fd840 msgr2=0x7f9adc104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 shutdown_connections 2024-04-26T18:14:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:28.251+0000 7f9ae0b4f700 1 -- 172.21.15.37:0/4005801666 wait complete. 2024-04-26T18:14:28.695 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:28.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:28.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:14:29.213 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:29.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:29 smithi037 ceph-mon[29171]: Added host smithi170 2024-04-26T18:14:29.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:29 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:29.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:29 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:30.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:30 smithi037 ceph-mon[29171]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.863+0000 7f9799442700 1 -- 172.21.15.37:0/1570093678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 msgr2=0x7f97941023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.863+0000 7f9799442700 1 --2- 172.21.15.37:0/1570093678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f97941023b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9784009910 tx=0x7f9784009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.863+0000 7f9799442700 1 -- 172.21.15.37:0/1570093678 shutdown_connections 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.863+0000 7f9799442700 1 --2- 172.21.15.37:0/1570093678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f97941023b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.863+0000 7f9799442700 1 -- 172.21.15.37:0/1570093678 >> 172.21.15.37:0/1570093678 conn(0x7f97940fd880 msgr2=0x7f97940ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 -- 172.21.15.37:0/1570093678 shutdown_connections 2024-04-26T18:14:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 -- 172.21.15.37:0/1570093678 wait complete. 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 Processor -- start 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 -- start start 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.864+0000 7f9799442700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f979410c250 con 0x7f9794101fd0 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9792ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9792ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34952/0 (socket says 172.21.15.37:34952) 2024-04-26T18:14:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9792ffd700 1 -- 172.21.15.37:0/230843592 learned_addr learned my addr 172.21.15.37:0/230843592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9792ffd700 1 -- 172.21.15.37:0/230843592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979419eb60 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9792ffd700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97840056d0 tx=0x7f97840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9784026070 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979419ee50 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978400bba0 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9784003920 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.865+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97941988c0 con 0x7f9794101fd0 2024-04-26T18:14:30.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.866+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9784029070 con 0x7f9794101fd0 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.866+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97941034b0 con 0x7f9794101fd0 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.866+0000 7f977bfff700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 0x7f977c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.866+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f978401e070 con 0x7f9794101fd0 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.866+0000 7f97927fc700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 0x7f977c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.867+0000 7f97927fc700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 0x7f977c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9788006fd0 tx=0x7f9788006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:30.871+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9784013070 con 0x7f9794101fd0 2024-04-26T18:14:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.089+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9794199c20 con 0x7f977c039650 2024-04-26T18:14:31.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.091+0000 7f977bfff700 1 -- 172.21.15.37:0/230843592 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f9794199c20 con 0x7f977c039650 2024-04-26T18:14:31.091 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:14:31.091 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-04-26T18:14:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 msgr2=0x7f977c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 0x7f977c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9788006fd0 tx=0x7f9788006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 msgr2=0x7f979419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97840056d0 tx=0x7f97840057b0 comp rx=0 tx=0).stop 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 shutdown_connections 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f977c039650 0x7f977c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 --2- 172.21.15.37:0/230843592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9794101fd0 0x7f979419e620 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 >> 172.21.15.37:0/230843592 conn(0x7f97940fd880 msgr2=0x7f979406b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 shutdown_connections 2024-04-26T18:14:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:31.093+0000 7f9799442700 1 -- 172.21.15.37:0/230843592 wait complete. 2024-04-26T18:14:31.788 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T18:14:31.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T18:14:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:32.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:31 smithi037 ceph-mon[29171]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:32.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:31 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:32.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:31 smithi037 ceph-mon[29171]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:33.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 -- 172.21.15.37:0/1171370582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 msgr2=0x7ffa541044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 --2- 172.21.15.37:0/1171370582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa541044b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009910 tx=0x7ffa44009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 -- 172.21.15.37:0/1171370582 shutdown_connections 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 --2- 172.21.15.37:0/1171370582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa541044b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 -- 172.21.15.37:0/1171370582 >> 172.21.15.37:0/1171370582 conn(0x7ffa540fd880 msgr2=0x7ffa540ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 -- 172.21.15.37:0/1171370582 shutdown_connections 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 -- 172.21.15.37:0/1171370582 wait complete. 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.422+0000 7ffa59cef700 1 Processor -- start 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa59cef700 1 -- start start 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa59cef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa59cef700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410d3d0 con 0x7ffa541040d0 2024-04-26T18:14:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34972/0 (socket says 172.21.15.37:34972) 2024-04-26T18:14:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa537fe700 1 -- 172.21.15.37:0/3720851145 learned_addr learned my addr 172.21.15.37:0/3720851145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa537fe700 1 -- 172.21.15.37:0/3720851145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa54103c20 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa537fe700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa440056d0 tx=0x7ffa440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa44026070 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa54102270 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44004370 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa440039b0 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.423+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa54102730 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.424+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa44029070 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.424+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5406bad0 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.424+0000 7ffa50ff9700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 0x7ffa3c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.425+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa4401e070 con 0x7ffa541040d0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.425+0000 7ffa52ffd700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 0x7ffa3c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.425+0000 7ffa52ffd700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 0x7ffa3c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006fd0 tx=0x7ffa48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.430+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa44013070 con 0x7ffa541040d0 2024-04-26T18:14:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.645+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ffa54103260 con 0x7ffa541040d0 2024-04-26T18:14:33.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.847+0000 7ffa50ff9700 1 -- 172.21.15.37:0/3720851145 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ffa4400bc10 con 0x7ffa541040d0 2024-04-26T18:14:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 msgr2=0x7ffa3c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 0x7ffa3c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006fd0 tx=0x7ffa48006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 msgr2=0x7ffa54101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa440056d0 tx=0x7ffa440057b0 comp rx=0 tx=0).stop 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 shutdown_connections 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ffa3c039650 0x7ffa3c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 --2- 172.21.15.37:0/3720851145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa541040d0 0x7ffa54101d30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 >> 172.21.15.37:0/3720851145 conn(0x7ffa540fd880 msgr2=0x7ffa54104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 shutdown_connections 2024-04-26T18:14:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:33.849+0000 7ffa59cef700 1 -- 172.21.15.37:0/3720851145 wait complete. 2024-04-26T18:14:33.851 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-26T18:14:33.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:33 smithi037 ceph-mon[29171]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:33.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:33 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:14:34.440 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-04-26T18:14:34.440 INFO:tasks.cephadm:Adding mon.b on smithi120 2024-04-26T18:14:34.440 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-04-26T18:14:34.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c' 2024-04-26T18:14:34.691 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:35.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:35.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:35.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.113+0000 7fd399874700 1 -- 172.21.15.170:0/1652932565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 msgr2=0x7fd394102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.113+0000 7fd399874700 1 --2- 172.21.15.170:0/1652932565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394102350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b0a0 tx=0x7fd38400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.113+0000 7fd399874700 1 -- 172.21.15.170:0/1652932565 shutdown_connections 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.113+0000 7fd399874700 1 --2- 172.21.15.170:0/1652932565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394102350 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.113+0000 7fd399874700 1 -- 172.21.15.170:0/1652932565 >> 172.21.15.170:0/1652932565 conn(0x7fd3940fd820 msgr2=0x7fd3940ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:36.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.114+0000 7fd399874700 1 -- 172.21.15.170:0/1652932565 shutdown_connections 2024-04-26T18:14:36.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.114+0000 7fd399874700 1 -- 172.21.15.170:0/1652932565 wait complete. 2024-04-26T18:14:36.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.114+0000 7fd399874700 1 Processor -- start 2024-04-26T18:14:36.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd399874700 1 -- start start 2024-04-26T18:14:36.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd399874700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:36.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd399874700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd39410c1f0 con 0x7fd394101f70 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd392ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.170:33888/0 (socket says 172.21.15.170:33888) 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd392ffd700 1 -- 172.21.15.170:0/1471797508 learned_addr learned my addr 172.21.15.170:0/1471797508 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.115+0000 7fd392ffd700 1 -- 172.21.15.170:0/1471797508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd394199c40 con 0x7fd394101f70 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd392ffd700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd384006130 tx=0x7fd3840097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384021020 con 0x7fd394101f70 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd394199ed0 con 0x7fd394101f70 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3840079b0 con 0x7fd394101f70 2024-04-26T18:14:36.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd384017040 con 0x7fd394101f70 2024-04-26T18:14:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.116+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39419a390 con 0x7fd394101f70 2024-04-26T18:14:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.117+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd394103450 con 0x7fd394101f70 2024-04-26T18:14:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.117+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd384024020 con 0x7fd394101f70 2024-04-26T18:14:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.117+0000 7fd398872700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 0x7fd38003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:36.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.117+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd38400f050 con 0x7fd394101f70 2024-04-26T18:14:36.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.117+0000 7fd3927fc700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 0x7fd38003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:36.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.118+0000 7fd3927fc700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 0x7fd38003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:36.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.123+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd38402b020 con 0x7fd394101f70 2024-04-26T18:14:36.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:35 smithi037 ceph-mon[29171]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:36.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:35 smithi037 ceph-mon[29171]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:36.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.329+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}) -- 0x7fd394064a80 con 0x7fd380039650 2024-04-26T18:14:36.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.337+0000 7fd398872700 1 -- 172.21.15.170:0/1471797508 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd394064a80 con 0x7fd380039650 2024-04-26T18:14:36.338 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 msgr2=0x7fd38003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 0x7fd38003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 msgr2=0x7fd394199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd384006130 tx=0x7fd3840097d0 comp rx=0 tx=0).stop 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 shutdown_connections 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd380039650 0x7fd38003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 --2- 172.21.15.170:0/1471797508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd394101f70 0x7fd394199700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 >> 172.21.15.170:0/1471797508 conn(0x7fd3940fd820 msgr2=0x7fd394104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 shutdown_connections 2024-04-26T18:14:36.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:36.339+0000 7fd399874700 1 -- 172.21.15.170:0/1471797508 wait complete. 2024-04-26T18:14:36.796 DEBUG:teuthology.orchestra.run.smithi120:mon.b> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T18:14:36.799 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T18:14:36.801 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:14:36.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:14:37.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c;count:3 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:37.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:37 smithi037 ceph-mon[29171]: Deploying daemon mon.c on smithi170 2024-04-26T18:14:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:38.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:38 smithi037 ceph-mon[29171]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:40.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.640+0000 7f3110f9a700 1 -- 172.21.15.170:0/1300261748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 msgr2=0x7f310c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:40.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.640+0000 7f3110f9a700 1 --2- 172.21.15.170:0/1300261748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c102380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009910 tx=0x7f30fc009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:40.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 -- 172.21.15.170:0/1300261748 shutdown_connections 2024-04-26T18:14:40.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 --2- 172.21.15.170:0/1300261748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c102380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:40.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 -- 172.21.15.170:0/1300261748 >> 172.21.15.170:0/1300261748 conn(0x7f310c0fd870 msgr2=0x7f310c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 -- 172.21.15.170:0/1300261748 shutdown_connections 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 -- 172.21.15.170:0/1300261748 wait complete. 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.641+0000 7f3110f9a700 1 Processor -- start 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f3110f9a700 1 -- start start 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f3110f9a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f3110f9a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f310c10c220 con 0x7f310c101fa0 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:40.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f310a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.170:57260/0 (socket says 172.21.15.170:57260) 2024-04-26T18:14:40.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.642+0000 7f310a59c700 1 -- 172.21.15.170:0/3225059105 learned_addr learned my addr 172.21.15.170:0/3225059105 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:14:40.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f310a59c700 1 -- 172.21.15.170:0/3225059105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c198c20 con 0x7f310c101fa0 2024-04-26T18:14:40.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f310a59c700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00b980 tx=0x7f30fc00ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:40.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc026070 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30fc005dc0 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c198f10 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30fc003a70 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.643+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c19cfb0 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.644+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f30fc029070 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.644+0000 7f30fb7fe700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 0x7f30f403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.644+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f30fc01e070 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.644+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f310c103480 con 0x7f310c101fa0 2024-04-26T18:14:40.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.645+0000 7f3109d9b700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 0x7f30f403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:40.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.645+0000 7f3109d9b700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 0x7f30f403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:40.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.650+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30fc030070 con 0x7f310c101fa0 2024-04-26T18:14:40.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:40 smithi037 ceph-mon[29171]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:40.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.942+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f310c064a80 con 0x7f310c101fa0 2024-04-26T18:14:40.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.942+0000 7f30fb7fe700 1 -- 172.21.15.170:0/3225059105 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f30fc013070 con 0x7f310c101fa0 2024-04-26T18:14:40.942 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:14:40.943 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:12:51.672210Z","created":"2024-04-26T18:12:51.672210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T18:14:40.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 msgr2=0x7f30f403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 0x7f30f403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3100006fd0 tx=0x7f3100006e40 comp rx=0 tx=0).stop 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 msgr2=0x7f310c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00b980 tx=0x7f30fc00ba60 comp rx=0 tx=0).stop 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 shutdown_connections 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f30f40396b0 0x7f30f403bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 --2- 172.21.15.170:0/3225059105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f310c101fa0 0x7f310c1986e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.944+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 >> 172.21.15.170:0/3225059105 conn(0x7f310c0fd870 msgr2=0x7f310c06b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.945+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 shutdown_connections 2024-04-26T18:14:40.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:40.945+0000 7f3110f9a700 1 -- 172.21.15.170:0/3225059105 wait complete. 2024-04-26T18:14:40.946 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-04-26T18:14:41.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 podman[32667]: 2024-04-26 18:14:41.06071786 +0000 UTC m=+0.363560255 container start c1339d7ac19a371a6beb585ea6c3e5a22113d5cf4acd9cd57464e1897645c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:14:41.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:14:41.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:14:41.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pidfile_write: ignore empty --pid-file 2024-04-26T18:14:41.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: load: jerasure load: lrc 2024-04-26T18:14:41.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Git sha 0 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: DB SUMMARY 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: DB Session ID: JEL1YEX4U9VMDLA4JR8I 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:14:41.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.env: 0x55baaa783cc0 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.info_log: 0x55baab891d80 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.statistics: (nil) 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.use_fsync: 0 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:14:41.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.db_log_dir: 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.wal_dir: 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:14:41.579 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.write_buffer_manager: 0x55baab8630e0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:14:41.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.unordered_write: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.row_cache: None 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.wal_filter: None 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.wal_compression: 0 2024-04-26T18:14:41.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:14:41.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:14:41.584 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_open_files: -1 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Compression algorithms supported: 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kZSTD supported: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:14:41.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.merge_operator: 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_filter: None 2024-04-26T18:14:41.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55baab891f40) 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2024-04-26T18:14:41.587 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x55baab8951f0 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2024-04-26T18:14:41.588 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:14:41.589 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression: NoCompression 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.num_levels: 7 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:14:41.590 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:14:41.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:14:41.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:14:41.593 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:14:41.594 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.ttl: 2592000 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:14:41.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5a96e2cf-9723-49d5-a6df-d6a689a93798 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155281092978, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155281093558, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714155281, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5a96e2cf-9723-49d5-a6df-d6a689a93798", "db_session_id": "JEL1YEX4U9VMDLA4JR8I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155281093661, "job": 1, "event": "recovery_finished"} 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55baab9b2000 2024-04-26T18:14:41.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: DB pointer 0x55baab99e000 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:14:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:14:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:14:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:14:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:14:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:14:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:14:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x55baab8951f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(???) e0 preinit fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2024-04-26T18:14:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Deploying cephadm binary to smithi170 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Added host smithi170 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.604 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:41.605 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c;count:3 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:41.606 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: Deploying daemon mon.c on smithi170 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 ceph-mon[32697]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 bash[32667]: c1339d7ac19a371a6beb585ea6c3e5a22113d5cf4acd9cd57464e1897645c0ec 2024-04-26T18:14:41.607 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:41 smithi170 systemd[1]: Started Ceph mon.c for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:14:42.647 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:14:42.648 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:14:42.890 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:14:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: mon.a calling monitor election 2024-04-26T18:14:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: mon.c calling monitor election 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: monmap epoch 2 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: last_changed 2024-04-26T18:14:41.118452+0000 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: min_mon_release 19 (squid) 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: election_strategy: 1 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: fsmap 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: mgrmap e13: x(active, since 49s) 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: overall HEALTH_OK 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: mon.a calling monitor election 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: mon.c calling monitor election 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: monmap epoch 2 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: last_changed 2024-04-26T18:14:41.118452+0000 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: min_mon_release 19 (squid) 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: election_strategy: 1 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: fsmap 2024-04-26T18:14:46.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: mgrmap e13: x(active, since 49s) 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: overall HEALTH_OK 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:46.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 -- 172.21.15.170:0/945984403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 msgr2=0x7fcbac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/945984403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbac005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4102e70 tx=0x7fcbbc009e80 comp rx=0 tx=0).stop 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 -- 172.21.15.170:0/945984403 shutdown_connections 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/945984403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbac005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 -- 172.21.15.170:0/945984403 >> 172.21.15.170:0/945984403 conn(0x7fcbc40fea30 msgr2=0x7fcbc4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 -- 172.21.15.170:0/945984403 shutdown_connections 2024-04-26T18:14:47.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.317+0000 7fcbcab5b700 1 -- 172.21.15.170:0/945984403 wait complete. 2024-04-26T18:14:47.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 Processor -- start 2024-04-26T18:14:47.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 -- start start 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbc4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc410ae90 con 0x7fcbac0035c0 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbcab5b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc410ad10 con 0x7fcbc4075c60 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50822/0 (socket says 172.21.15.170:50822) 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.318+0000 7fcbc3fff700 1 -- 172.21.15.170:0/4074858965 learned_addr learned my addr 172.21.15.170:0/4074858965 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc3fff700 1 -- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 msgr2=0x7fcbc40760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc3fff700 1 -- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 msgr2=0x7fcbc40760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc3fff700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc88f7700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbc4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc3fff700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc88f7700 1 -- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 msgr2=0x7fcbc40760e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:14:47.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc88f7700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:47.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc88f7700 1 -- 172.21.15.170:0/4074858965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc4077640 con 0x7fcbac0035c0 2024-04-26T18:14:47.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc88f7700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbc4075720 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4102e70 tx=0x7fcbbc017750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:47.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbbc02a470 con 0x7fcbac0035c0 2024-04-26T18:14:47.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc41a4cb0 con 0x7fcbac0035c0 2024-04-26T18:14:47.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbbc025070 con 0x7fcbac0035c0 2024-04-26T18:14:47.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.319+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbbc0219c0 con 0x7fcbac0035c0 2024-04-26T18:14:47.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.320+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41a51a0 con 0x7fcbac0035c0 2024-04-26T18:14:47.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.321+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcbbc01e070 con 0x7fcbac0035c0 2024-04-26T18:14:47.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.321+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc4104680 con 0x7fcbac0035c0 2024-04-26T18:14:47.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.321+0000 7fcbc1ffb700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 0x7fcbb003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:47.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.321+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcbbc01c070 con 0x7fcbac0035c0 2024-04-26T18:14:47.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.321+0000 7fcbc3fff700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 0x7fcbb003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:47.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.322+0000 7fcbc3fff700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 0x7fcbb003b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009180 tx=0x7fcbb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:47.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.327+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbbc029c90 con 0x7fcbac0035c0 2024-04-26T18:14:47.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:47 smithi037 ceph-mon[29171]: Deploying daemon mon.b on smithi120 2024-04-26T18:14:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:47 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:47 smithi170 ceph-mon[32697]: Deploying daemon mon.b on smithi120 2024-04-26T18:14:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:47 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:47.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.608+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcbc402e490 con 0x7fcbac0035c0 2024-04-26T18:14:47.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.608+0000 7fcbc1ffb700 1 -- 172.21.15.170:0/4074858965 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fcbbc01f3d0 con 0x7fcbac0035c0 2024-04-26T18:14:47.608 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:14:47.608 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:14:41.118452Z","created":"2024-04-26T18:12:51.672210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:14:47.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 msgr2=0x7fcbb003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:47.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 0x7fcbb003b8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009180 tx=0x7fcbb8008040 comp rx=0 tx=0).stop 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 msgr2=0x7fcbc4075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbc4075720 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4102e70 tx=0x7fcbbc017750 comp rx=0 tx=0).stop 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 shutdown_connections 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcbb0039410 0x7fcbb003b8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbac0035c0 0x7fcbc4075720 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 --2- 172.21.15.170:0/4074858965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbc4075c60 0x7fcbc40760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 >> 172.21.15.170:0/4074858965 conn(0x7fcbc40fea30 msgr2=0x7fcbc40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 shutdown_connections 2024-04-26T18:14:47.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:47.610+0000 7fcbcab5b700 1 -- 172.21.15.170:0/4074858965 wait complete. 2024-04-26T18:14:47.612 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-04-26T18:14:48.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:48 smithi037 ceph-mon[29171]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:48 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/4074858965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:48 smithi170 ceph-mon[32697]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:48 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/4074858965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:49.212 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:14:49.212 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:14:49.456 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:14:49.734 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 podman[32225]: 2024-04-26 18:14:49.554381742 +0000 UTC m=+0.018509105 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:14:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 podman[32225]: 2024-04-26 18:14:49.832859225 +0000 UTC m=+0.296986558 container init 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T18:14:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 podman[32225]: 2024-04-26 18:14:49.836337684 +0000 UTC m=+0.300465016 container start 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T18:14:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pidfile_write: ignore empty --pid-file 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: load: jerasure load: lrc 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Git sha 0 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: DB SUMMARY 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: DB Session ID: WOWGVYJ3TNN1V3CACQAP 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:14:50.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.env: 0x55a9b7b79cc0 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:14:50.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.info_log: 0x55a9b8ef5d80 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.statistics: (nil) 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.use_fsync: 0 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:14:50.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.db_log_dir: 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.wal_dir: 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:14:50.007 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.write_buffer_manager: 0x55a9b8ec70e0 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:14:50.008 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.unordered_write: 0 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.row_cache: None 2024-04-26T18:14:50.009 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.wal_filter: None 2024-04-26T18:14:50.011 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:14:50.011 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:14:50.011 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:14:50.011 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:14:50.011 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.wal_compression: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:14:50.012 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_open_files: -1 2024-04-26T18:14:50.013 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Compression algorithms supported: 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kZSTD supported: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:14:50.014 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.merge_operator: 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_filter: None 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:14:50.015 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a9b8ef5f40) 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_type: 0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_index_type: 0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_shortening: 1 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: checksum: 4 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: no_block_cache: 0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache: 0x55a9b8ef91f0 2024-04-26T18:14:50.016 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_options: 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: capacity : 536870912 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_shard_bits : 4 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: strict_capacity_limit : 0 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_compressed: (nil) 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: persistent_cache: (nil) 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size: 4096 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size_deviation: 10 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_restart_interval: 16 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_block_restart_interval: 1 2024-04-26T18:14:50.017 INFO:journalctl@ceph.mon.b.smithi120.stdout: metadata_block_size: 4096 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: partition_filters: 0 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: use_delta_encoding: 1 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: filter_policy: bloomfilter 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: whole_key_filtering: 1 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: verify_compression: 0 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: format_version: 5 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_index_compression: 1 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_align: 0 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: max_auto_readahead_size: 262144 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: prepopulate_block_cache: 0 2024-04-26T18:14:50.018 INFO:journalctl@ceph.mon.b.smithi120.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression: NoCompression 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.num_levels: 7 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:14:50.019 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:14:50.020 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:14:50.021 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:14:50.022 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:14:50.023 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:14:50.024 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.ttl: 2592000 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:14:50.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T18:14:50.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 874d7c26-50eb-4b21-887e-b877c5bd8154 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155289877184, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155289877621, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714155289, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "874d7c26-50eb-4b21-887e-b877c5bd8154", "db_session_id": "WOWGVYJ3TNN1V3CACQAP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155289877700, "job": 1, "event": "recovery_finished"} 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a9b9016000 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: DB pointer 0x55a9b9002000 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:14:50.027 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** DB Stats ** 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:14:50.028 INFO:journalctl@ceph.mon.b.smithi120.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:14:50.029 INFO:journalctl@ceph.mon.b.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache BinnedLRUCache@0x55a9b8ef91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.030 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: using public_addr v2:172.21.15.120:0/0 -> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: starting mon.b rank -1 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(???) e0 preinit fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: e1 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: legacy client fscid: -1 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-04-26T18:14:50.031 INFO:journalctl@ceph.mon.b.smithi120.stdout: No filesystems configured 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.032 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:50.033 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Deploying cephadm binary to smithi170 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:50.034 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Added host smithi170 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3720851145' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.035 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.036 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c;count:3 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Deploying daemon mon.c on smithi170 2024-04-26T18:14:50.037 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.a calling monitor election 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.c calling monitor election 2024-04-26T18:14:50.038 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: monmap epoch 2 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: last_changed 2024-04-26T18:14:41.118452+0000 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: min_mon_release 19 (squid) 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: election_strategy: 1 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:50.039 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: fsmap 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mgrmap e13: x(active, since 49s) 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: overall HEALTH_OK 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: Deploying daemon mon.b on smithi120 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:50.040 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:50.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/4074858965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:50.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 ceph-mon[32244]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T18:14:50.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 bash[32225]: 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb 2024-04-26T18:14:50.041 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:49 smithi120 systemd[1]: Started Ceph mon.b for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: mon.c calling monitor election 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: mon.a calling monitor election 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: monmap epoch 3 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: min_mon_release 19 (squid) 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: election_strategy: 1 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: fsmap 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: mgrmap e13: x(active, since 58s) 2024-04-26T18:14:55.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: overall HEALTH_OK 2024-04-26T18:14:55.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:55.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:55.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 -- 172.21.15.170:0/1762945580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 msgr2=0x7f6ed0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/1762945580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ed0005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8102e90 tx=0x7f6ed800fe60 comp rx=0 tx=0).stop 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 -- 172.21.15.170:0/1762945580 shutdown_connections 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/1762945580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ed0005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 -- 172.21.15.170:0/1762945580 >> 172.21.15.170:0/1762945580 conn(0x7f6ee80fea10 msgr2=0x7f6ee8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 -- 172.21.15.170:0/1762945580 shutdown_connections 2024-04-26T18:14:55.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.237+0000 7f6eeea5f700 1 -- 172.21.15.170:0/1762945580 wait complete. 2024-04-26T18:14:55.238 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: mon.c calling monitor election 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: mon.a calling monitor election 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: monmap epoch 3 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:55.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: min_mon_release 19 (squid) 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: election_strategy: 1 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: fsmap 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: mgrmap e13: x(active, since 58s) 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: overall HEALTH_OK 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:55.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.238+0000 7f6eeea5f700 1 Processor -- start 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 -- start start 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee810ae50 con 0x7f6ed00035c0 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee810acd0 con 0x7f6ee8075c60 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eeea5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee810afd0 con 0x7f6ee81a4cf0 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6ee7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6eecffc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:55.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6ee7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.170:57306/0 (socket says 172.21.15.170:57306) 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6ee7fff700 1 -- 172.21.15.170:0/3027383336 learned_addr learned my addr 172.21.15.170:0/3027383336 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.239+0000 7f6ee77fe700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eecffc700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 msgr2=0x7f6ee81a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eecffc700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 msgr2=0x7f6ee81a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eecffc700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eecffc700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee77fe700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 msgr2=0x7f6ee80760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee77fe700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 msgr2=0x7f6ee80760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee77fe700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 msgr2=0x7f6ee81a70e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 msgr2=0x7f6ee80760e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee77fe700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:14:55.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 -- 172.21.15.170:0/3027383336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81a77b0 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee7fff700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8003ca0 tx=0x7f6ed8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed802b020 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ed801d440 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81a7a40 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8022040 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.240+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81a7fd0 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.241+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6ed802e020 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.242+0000 7f6ee57fa700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 0x7f6ed403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.242+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6ed804f1d0 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.242+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee8064a80 con 0x7f6ed00035c0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.242+0000 7f6ee77fe700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 0x7f6ed403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.242+0000 7f6ee77fe700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 0x7f6ed403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009850 tx=0x7f6edc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:55.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.247+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ed8035020 con 0x7f6ed00035c0 2024-04-26T18:14:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.538+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6ee81a82a0 con 0x7f6ed00035c0 2024-04-26T18:14:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.539+0000 7f6ee57fa700 1 -- 172.21.15.170:0/3027383336 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f6ed8020120 con 0x7f6ed00035c0 2024-04-26T18:14:55.539 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-26T18:14:55.539 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:14:49.900195Z","created":"2024-04-26T18:12:51.672210Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:14:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 msgr2=0x7f6ed403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 0x7f6ed403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009850 tx=0x7f6edc008040 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 msgr2=0x7f6ee8075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8003ca0 tx=0x7f6ed8003d80 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 shutdown_connections 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ed4039ae0 0x7f6ed403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee81a4cf0 0x7f6ee81a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed00035c0 0x7f6ee8075720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 --2- 172.21.15.170:0/3027383336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ee8075c60 0x7f6ee80760e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 >> 172.21.15.170:0/3027383336 conn(0x7f6ee80fea10 msgr2=0x7f6ee8106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 shutdown_connections 2024-04-26T18:14:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:14:55.541+0000 7f6eeea5f700 1 -- 172.21.15.170:0/3027383336 wait complete. 2024-04-26T18:14:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-04-26T18:14:56.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:56.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:56.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:56.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:56.123 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/3027383336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:56.124 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:56.146 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T18:14:56.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/3027383336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:56.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: mon.c calling monitor election 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: mon.a calling monitor election 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:14:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: monmap epoch 3 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:57.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: min_mon_release 19 (squid) 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: election_strategy: 1 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: fsmap 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: mgrmap e13: x(active, since 58s) 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: overall HEALTH_OK 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:57.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:57.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:57.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:57.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/3027383336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:14:57.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 -- 172.21.15.37:0/201643158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 msgr2=0x7f4318103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 --2- 172.21.15.37:0/201643158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f4318103470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4308009910 tx=0x7f4308009c20 comp rx=0 tx=0).stop 2024-04-26T18:14:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 -- 172.21.15.37:0/201643158 shutdown_connections 2024-04-26T18:14:58.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 --2- 172.21.15.37:0/201643158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f4318103470 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 -- 172.21.15.37:0/201643158 >> 172.21.15.37:0/201643158 conn(0x7f43180fe940 msgr2=0x7f4318100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 -- 172.21.15.37:0/201643158 shutdown_connections 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.004+0000 7f431e20d700 1 -- 172.21.15.37:0/201643158 wait complete. 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.005+0000 7f431e20d700 1 Processor -- start 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.005+0000 7f431e20d700 1 -- start start 2024-04-26T18:14:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.005+0000 7f431e20d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.005+0000 7f431e20d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 0x7f4318198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.005+0000 7f431e20d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 0x7f4318199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f431e20d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431810d310 con 0x7f4318103090 2024-04-26T18:14:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f431e20d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f431810d190 con 0x7f4318198e50 2024-04-26T18:14:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f431e20d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f431810d490 con 0x7f431819ec90 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59180/0 (socket says 172.21.15.37:59180) 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 -- 172.21.15.37:0/3416619386 learned_addr learned my addr 172.21.15.37:0/3416619386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f4316ffd700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 0x7f4318198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f4317fff700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 0x7f4318199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 -- 172.21.15.37:0/3416619386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 msgr2=0x7f4318198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 0x7f4318198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 -- 172.21.15.37:0/3416619386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 msgr2=0x7f4318199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 0x7f4318199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 -- 172.21.15.37:0/3416619386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181a5620 con 0x7f4318103090 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f4316ffd700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 0x7f4318198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:14:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f4317fff700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 0x7f4318199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f43177fe700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4308004f80 tx=0x7f4308005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308026070 con 0x7f4318103090 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.006+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181a5850 con 0x7f4318103090 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.007+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430800b6f0 con 0x7f4318103090 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.007+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308003970 con 0x7f4318103090 2024-04-26T18:14:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.007+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a5de0 con 0x7f4318103090 2024-04-26T18:14:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.007+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4308029070 con 0x7f4318103090 2024-04-26T18:14:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.008+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4318064a80 con 0x7f4318103090 2024-04-26T18:14:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.008+0000 7f4314ff9700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 0x7f430003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.008+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f430804d4d0 con 0x7f4318103090 2024-04-26T18:14:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.008+0000 7f4316ffd700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 0x7f430003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.008+0000 7f4316ffd700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 0x7f430003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f430c009910 tx=0x7f430c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.013+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4308013070 con 0x7f4318103090 2024-04-26T18:14:58.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mon.b calling monitor election 2024-04-26T18:14:58.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mon.b calling monitor election 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mon.a calling monitor election 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mon.c calling monitor election 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: monmap epoch 3 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: min_mon_release 19 (squid) 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: election_strategy: 1 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:58.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: fsmap 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: mgrmap e13: x(active, since 60s) 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: overall HEALTH_OK 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:14:58.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:58.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.225+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f43181a6010 con 0x7f4318103090 2024-04-26T18:14:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.226+0000 7f4314ff9700 1 -- 172.21.15.37:0/3416619386 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4308030090 con 0x7f4318103090 2024-04-26T18:14:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:58.226 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-26T18:14:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 msgr2=0x7f430003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 0x7f430003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f430c009910 tx=0x7f430c008040 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 msgr2=0x7f431819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4308004f80 tx=0x7f4308005060 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 shutdown_connections 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4300039ae0 0x7f430003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4318198e50 0x7f4318199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4318103090 0x7f431819e750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 --2- 172.21.15.37:0/3416619386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431819ec90 0x7f4318198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.228+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 >> 172.21.15.37:0/3416619386 conn(0x7f43180fe940 msgr2=0x7f4318104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.229+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 shutdown_connections 2024-04-26T18:14:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:14:58.229+0000 7f431e20d700 1 -- 172.21.15.37:0/3416619386 wait complete. 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mon.b calling monitor election 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mon.b calling monitor election 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mon.a calling monitor election 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mon.c calling monitor election 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: monmap epoch 3 2024-04-26T18:14:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: min_mon_release 19 (squid) 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: election_strategy: 1 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: fsmap 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: mgrmap e13: x(active, since 60s) 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: overall HEALTH_OK 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:58.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:14:58.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mon.b calling monitor election 2024-04-26T18:14:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mon.b calling monitor election 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mon.a calling monitor election 2024-04-26T18:14:58.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mon.c calling monitor election 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: monmap epoch 3 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: last_changed 2024-04-26T18:14:49.900195+0000 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: created 2024-04-26T18:12:51.672210+0000 2024-04-26T18:14:58.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: min_mon_release 19 (squid) 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: election_strategy: 1 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: fsmap 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:14:58.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: mgrmap e13: x(active, since 60s) 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: overall HEALTH_OK 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.330 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:14:58.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:14:58.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:14:58.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:58 smithi037 ceph-mon[29171]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:14:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:58 smithi037 ceph-mon[29171]: Reconfiguring daemon mon.a on smithi037 2024-04-26T18:14:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:58 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3416619386' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:59.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:58 smithi120 ceph-mon[32244]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:14:59.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:58 smithi120 ceph-mon[32244]: Reconfiguring daemon mon.a on smithi037 2024-04-26T18:14:59.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:59.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:58 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3416619386' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:58 smithi170 ceph-mon[32697]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:14:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:58 smithi170 ceph-mon[32697]: Reconfiguring daemon mon.a on smithi037 2024-04-26T18:14:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:14:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:58 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3416619386' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:59.357 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T18:14:59.358 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:59.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:14:59.386 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:14:59.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:59.451 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:14:59.451 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:14:59.484 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:14:59.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:59.550 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:14:59.551 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:14:59.582 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:14:59.582 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:14:59.650 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-04-26T18:14:59.650 INFO:tasks.cephadm:Adding mgr.y on smithi120 2024-04-26T18:14:59.650 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-04-26T18:14:59.650 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi120=y;smithi170=z' 2024-04-26T18:14:59.925 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:15:00.154 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:00.155 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.155 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.155 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:00.155 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:00.155 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:14:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:00.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:00.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:00.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:00.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:14:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:00.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:00.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:00.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:00.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:00.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:14:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:01.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:00 smithi120 ceph-mon[32244]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:15:01.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:00 smithi120 ceph-mon[32244]: Reconfiguring daemon mon.b on smithi120 2024-04-26T18:15:01.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:00 smithi170 ceph-mon[32697]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:15:01.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:00 smithi170 ceph-mon[32697]: Reconfiguring daemon mon.b on smithi120 2024-04-26T18:15:01.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 -- 172.21.15.170:0/3875392650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 msgr2=0x7f79f0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:01.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 --2- 172.21.15.170:0/3875392650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f0102290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009910 tx=0x7f79e8009c20 comp rx=0 tx=0).stop 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 -- 172.21.15.170:0/3875392650 shutdown_connections 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 --2- 172.21.15.170:0/3875392650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f0102290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 -- 172.21.15.170:0/3875392650 >> 172.21.15.170:0/3875392650 conn(0x7f79f00fd460 msgr2=0x7f79f00ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 -- 172.21.15.170:0/3875392650 shutdown_connections 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.225+0000 7f79f8737700 1 -- 172.21.15.170:0/3875392650 wait complete. 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 Processor -- start 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 -- start start 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f010a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79f01088c0 0x7f79f0108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0107d70 con 0x7f79f0101e70 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0107bf0 con 0x7f79f01088c0 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f8737700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79f0107ef0 con 0x7f79f0109280 2024-04-26T18:15:01.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f6cd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f6cd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57670/0 (socket says 172.21.15.170:57670) 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f6cd4700 1 -- 172.21.15.170:0/2914219331 learned_addr learned my addr 172.21.15.170:0/2914219331 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.226+0000 7f79f64d3700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f010a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f5cd2700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79f01088c0 0x7f79f0108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 -- 172.21.15.170:0/2914219331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79f01088c0 msgr2=0x7f79f0108d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79f01088c0 0x7f79f0108d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 -- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 msgr2=0x7f79f010a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f010a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 -- 172.21.15.170:0/2914219331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f01a2dc0 con 0x7f79f0109280 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f64d3700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f6cd4700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00ba50 tx=0x7f79ec00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec019070 con 0x7f79f0109280 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ec014440 con 0x7f79f0109280 2024-04-26T18:15:01.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f01a30b0 con 0x7f79f0109280 2024-04-26T18:15:01.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00ce60 con 0x7f79f0109280 2024-04-26T18:15:01.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.227+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01a3670 con 0x7f79f0109280 2024-04-26T18:15:01.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.228+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f79ec01c070 con 0x7f79f0109280 2024-04-26T18:15:01.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.228+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f0064a80 con 0x7f79f0109280 2024-04-26T18:15:01.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.228+0000 7f79e77fe700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 0x7f79dc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:01.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.228+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79ec04bdf0 con 0x7f79f0109280 2024-04-26T18:15:01.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.228+0000 7f79f64d3700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 0x7f79dc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:01.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.229+0000 7f79f64d3700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 0x7f79dc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79e8006010 tx=0x7f79e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:01.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:00 smithi037 ceph-mon[29171]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:15:01.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:00 smithi037 ceph-mon[29171]: Reconfiguring daemon mon.b on smithi120 2024-04-26T18:15:01.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.233+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79ec012740 con 0x7f79f0109280 2024-04-26T18:15:01.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.441+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi120=y;smithi170=z", "target": ["mon-mgr", ""]}) -- 0x7f79f00638f0 con 0x7f79dc039ae0 2024-04-26T18:15:01.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.447+0000 7f79e77fe700 1 -- 172.21.15.170:0/2914219331 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f79f00638f0 con 0x7f79dc039ae0 2024-04-26T18:15:01.447 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.449+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 msgr2=0x7f79dc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.449+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 0x7f79dc03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79e8006010 tx=0x7f79e801a040 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.449+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 msgr2=0x7f79f01a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.449+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00ba50 tx=0x7f79ec00bd60 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 shutdown_connections 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f79dc039ae0 0x7f79dc03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79f0109280 0x7f79f01a2800 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79f0101e70 0x7f79f010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 --2- 172.21.15.170:0/2914219331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f79f01088c0 0x7f79f0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 >> 172.21.15.170:0/2914219331 conn(0x7f79f00fd460 msgr2=0x7f79f018f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 shutdown_connections 2024-04-26T18:15:01.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:01.450+0000 7f79f8737700 1 -- 172.21.15.170:0/2914219331 wait complete. 2024-04-26T18:15:02.094 DEBUG:teuthology.orchestra.run.smithi120:mgr.y> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T18:15:02.097 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T18:15:02.100 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T18:15:02.100 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:15:02.100 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:15:02.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:15:02.120 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 18:14:06.890180425 +0000 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 18:09:28.023692603 +0000 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 18:09:28.023692603 +0000 2024-04-26T18:15:02.180 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 18:09:28.023692603 +0000 2024-04-26T18:15:02.181 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:15:02.249 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T18:15:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T18:15:02.250 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000236553 s, 2.2 MB/s 2024-04-26T18:15:02.251 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:15:02.310 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:02.368 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 18:14:06.918180037 +0000 2024-04-26T18:15:02.369 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 18:09:28.385687344 +0000 2024-04-26T18:15:02.369 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 18:09:28.385687344 +0000 2024-04-26T18:15:02.369 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 18:09:28.385687344 +0000 2024-04-26T18:15:02.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:15:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T18:15:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T18:15:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00020926 s, 2.4 MB/s 2024-04-26T18:15:02.436 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:15:02.494 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 18:14:06.944179677 +0000 2024-04-26T18:15:02.552 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 18:09:28.736682246 +0000 2024-04-26T18:15:02.553 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 18:09:28.736682246 +0000 2024-04-26T18:15:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 18:09:28.736682246 +0000 2024-04-26T18:15:02.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi120=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: Saving service mgr spec with placement smithi037=x;smithi120=y;smithi170=z;count:3 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:02.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T18:15:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T18:15:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000224387 s, 2.3 MB/s 2024-04-26T18:15:02.619 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:15:02.676 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-26T18:15:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:15:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 18:14:06.973179275 +0000 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 18:09:29.094677045 +0000 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 18:09:29.094677045 +0000 2024-04-26T18:15:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 18:09:29.094677045 +0000 2024-04-26T18:15:02.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:15:02.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:02.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi120=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:02.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: Saving service mgr spec with placement smithi037=x;smithi120=y;smithi170=z;count:3 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T18:15:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T18:15:02.798 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00021443 s, 2.4 MB/s 2024-04-26T18:15:02.800 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi120=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: Saving service mgr spec with placement smithi037=x;smithi120=y;smithi170=z;count:3 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:15:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:15:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:02.857 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T18:15:02.857 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:15:02.876 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:15:02.876 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-04-26T18:15:02.932 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 18:14:49.341370265 +0000 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 18:09:46.082080861 +0000 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 18:09:46.082080861 +0000 2024-04-26T18:15:02.933 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 18:09:46.082080861 +0000 2024-04-26T18:15:02.933 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:15:02.998 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T18:15:02.998 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T18:15:02.998 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000284253 s, 1.8 MB/s 2024-04-26T18:15:03.000 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:15:03.057 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 18:14:49.341370265 +0000 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 18:09:46.441072317 +0000 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 18:09:46.441072317 +0000 2024-04-26T18:15:03.114 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 18:09:46.441072317 +0000 2024-04-26T18:15:03.115 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:15:03.178 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T18:15:03.178 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T18:15:03.178 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00024383 s, 2.1 MB/s 2024-04-26T18:15:03.179 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:15:03.238 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 18:14:49.342370242 +0000 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 18:09:46.778064297 +0000 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 18:09:46.778064297 +0000 2024-04-26T18:15:03.294 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 18:09:46.778064297 +0000 2024-04-26T18:15:03.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:15:03.356 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T18:15:03.356 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T18:15:03.356 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000211599 s, 2.4 MB/s 2024-04-26T18:15:03.357 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:15:03.412 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-04-26T18:15:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:15:03.468 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:03.468 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T18:15:03.468 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:03.469 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:03.469 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 18:14:49.342370242 +0000 2024-04-26T18:15:03.469 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 18:09:47.119056181 +0000 2024-04-26T18:15:03.469 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 18:09:47.119056181 +0000 2024-04-26T18:15:03.469 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 18:09:47.119056181 +0000 2024-04-26T18:15:03.469 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:15:03.531 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T18:15:03.531 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T18:15:03.531 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000215084 s, 2.4 MB/s 2024-04-26T18:15:03.532 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:15:03.587 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-26T18:15:03.587 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:15:03.605 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:15:03.605 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-04-26T18:15:03.662 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:15:03.662 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:03.662 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-26 18:14:40.345813976 +0000 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-26 18:09:40.056495186 +0000 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-26 18:09:40.056495186 +0000 2024-04-26T18:15:03.663 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-26 18:09:40.056495186 +0000 2024-04-26T18:15:03.663 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:15:03.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:03 smithi170 ceph-mon[32697]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:15:03.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:03 smithi170 ceph-mon[32697]: Reconfiguring daemon mon.c on smithi170 2024-04-26T18:15:03.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-26T18:15:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-26T18:15:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000114697 s, 4.5 MB/s 2024-04-26T18:15:03.730 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:15:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:03 smithi037 ceph-mon[29171]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:15:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:03 smithi037 ceph-mon[29171]: Reconfiguring daemon mon.c on smithi170 2024-04-26T18:15:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:03.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:03 smithi120 ceph-mon[32244]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:15:03.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:03 smithi120 ceph-mon[32244]: Reconfiguring daemon mon.c on smithi170 2024-04-26T18:15:03.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:03.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:03.786 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-04-26T18:15:03.840 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-26 18:14:40.345813976 +0000 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-26 18:09:40.398486601 +0000 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-26 18:09:40.398486601 +0000 2024-04-26T18:15:03.841 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-26 18:09:40.398486601 +0000 2024-04-26T18:15:03.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:15:03.903 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-26T18:15:03.903 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-26T18:15:03.903 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000112497 s, 4.6 MB/s 2024-04-26T18:15:03.904 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:15:03.960 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-04-26T18:15:04.015 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:15:04.015 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:04.015 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-26 18:14:40.346813952 +0000 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-26 18:09:40.720478517 +0000 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-26 18:09:40.720478517 +0000 2024-04-26T18:15:04.016 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-26 18:09:40.720478517 +0000 2024-04-26T18:15:04.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:15:04.080 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-26T18:15:04.080 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-26T18:15:04.081 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.3039e-05 s, 5.5 MB/s 2024-04-26T18:15:04.081 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:15:04.137 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-04-26T18:15:04.193 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:15:04.193 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:15:04.193 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T18:15:04.193 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:15:04.194 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:15:04.194 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-26 18:14:40.346813952 +0000 2024-04-26T18:15:04.194 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-26 18:09:41.043470409 +0000 2024-04-26T18:15:04.194 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-26 18:09:41.043470409 +0000 2024-04-26T18:15:04.194 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-26 18:09:41.043470409 +0000 2024-04-26T18:15:04.194 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:15:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-26T18:15:04.256 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-26T18:15:04.257 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.1469e-05 s, 5.6 MB/s 2024-04-26T18:15:04.258 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:15:04.313 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-26T18:15:04.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:15:04.562 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:04.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:04.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:04.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:04.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:04.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:04.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:05.723 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:05 smithi037 ceph-mon[29171]: Deploying daemon mgr.z on smithi170 2024-04-26T18:15:05.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:05 smithi120 ceph-mon[32244]: Deploying daemon mgr.z on smithi170 2024-04-26T18:15:05.767 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:05 smithi170 ceph-mon[32697]: Deploying daemon mgr.z on smithi170 2024-04-26T18:15:05.767 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 systemd[1]: Starting Ceph mgr.z for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 podman[34538]: 2024-04-26 18:15:05.767556718 +0000 UTC m=+0.100564164 container create 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 podman[34538]: 2024-04-26 18:15:05.685079033 +0000 UTC m=+0.018086444 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 podman[34538]: 2024-04-26 18:15:05.969751008 +0000 UTC m=+0.302758408 container init 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 podman[34538]: 2024-04-26 18:15:05.972360775 +0000 UTC m=+0.305368182 container start 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:05 smithi170 ceph-mgr[34556]: -- 172.21.15.170:0/2804901279 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e47ab05a0 con 0x564e47a84c00 2024-04-26T18:15:06.036 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 bash[34538]: 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 2024-04-26T18:15:06.314 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 systemd[1]: Started Ceph mgr.z for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:15:06.314 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:06.130+0000 7fe90fd62200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:15:06.314 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:06.314+0000 7fe90fd62200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:15:06.389 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:15:06.390 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:15:06.390 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T18:15:06.390 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T18:15:06.390 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110968 s, 945 MB/s 2024-04-26T18:15:06.390 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T18:15:06.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:06.416+0000 7fe90fd62200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:15:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:06.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:06.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:07.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-26T18:15:07.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:06.725+0000 7fe90fd62200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:15:07.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:06.809+0000 7fe90fd62200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:15:07.256 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:15:07.648 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:07 smithi037 ceph-mon[29171]: Deploying daemon mgr.y on smithi120 2024-04-26T18:15:07.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:07 smithi120 ceph-mon[32244]: Deploying daemon mgr.y on smithi120 2024-04-26T18:15:07.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:07 smithi170 ceph-mon[32697]: Deploying daemon mgr.y on smithi120 2024-04-26T18:15:08.230 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 systemd[1]: Starting Ceph mgr.y for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:15:08.230 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 podman[33612]: 2024-04-26 18:15:08.13352656 +0000 UTC m=+0.017747703 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:15:08.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:08.070+0000 7fe90fd62200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:15:08.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:08.242+0000 7fe90fd62200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:15:08.487 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:08 smithi120 ceph-mon[32244]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:08.488 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 podman[33612]: 2024-04-26 18:15:08.244172752 +0000 UTC m=+0.128393926 container create a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:15:08.488 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 podman[33612]: 2024-04-26 18:15:08.421838196 +0000 UTC m=+0.306059331 container init a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:15:08.488 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 podman[33612]: 2024-04-26 18:15:08.424279583 +0000 UTC m=+0.308500715 container start a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T18:15:08.488 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 ceph-mgr[33631]: -- 172.21.15.120:0/2145544588 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56128a9425a0 con 0x56128a916800 2024-04-26T18:15:08.488 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 bash[33612]: a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 2024-04-26T18:15:08.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:08 smithi037 ceph-mon[29171]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:08.706 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-mon[32697]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:08.706 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:08.418+0000 7fe90fd62200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:15:08.706 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:08.706+0000 7fe90fd62200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:15:08.752 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 systemd[1]: Started Ceph mgr.y for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:15:08.752 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:08.575+0000 7f7648065200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:15:08.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 -- 172.21.15.37:0/652066376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb800105040 msgr2=0x7fb80010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 --2- 172.21.15.37:0/652066376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb800105040 0x7fb80010b8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b0d0 tx=0x7fb7f800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 -- 172.21.15.37:0/652066376 shutdown_connections 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 --2- 172.21.15.37:0/652066376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb800105040 0x7fb80010b8f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 --2- 172.21.15.37:0/652066376 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8001034a0 0x7fb8001038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 --2- 172.21.15.37:0/652066376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 -- 172.21.15.37:0/652066376 >> 172.21.15.37:0/652066376 conn(0x7fb8000fea40 msgr2=0x7fb800100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 -- 172.21.15.37:0/652066376 shutdown_connections 2024-04-26T18:15:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.796+0000 7fb80495a700 1 -- 172.21.15.37:0/652066376 wait complete. 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 Processor -- start 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 -- start start 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 0x7fb8000771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010ef20 con 0x7fb8001034a0 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010eda0 con 0x7fb800105040 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.797+0000 7fb80495a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010f0a0 con 0x7fb8001046a0 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fe59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 0x7fb8000771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:08.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:44862/0 (socket says 172.21.15.37:44862) 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:41834/0 (socket says 172.21.15.37:41834) 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fe59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 0x7fb8000771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49452/0 (socket says 172.21.15.37:49452) 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fdd9b700 1 -- 172.21.15.37:0/1280750126 learned_addr learned my addr 172.21.15.37:0/1280750126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 -- 172.21.15.37:0/1280750126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 msgr2=0x7fb800075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 -- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 msgr2=0x7fb8000771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 0x7fb8000771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 -- 172.21.15.37:0/1280750126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800076600 con 0x7fb800105040 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7fed9d700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b770 tx=0x7fb7f8004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8021070 con 0x7fb800105040 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f8007b80 con 0x7fb800105040 2024-04-26T18:15:08.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.798+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001a3630 con 0x7fb800105040 2024-04-26T18:15:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.799+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f801b6b0 con 0x7fb800105040 2024-04-26T18:15:08.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.799+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001a3a80 con 0x7fb800105040 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.800+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb7f8024070 con 0x7fb800105040 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.800+0000 7fb7ef7fe700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 0x7fb7e803bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.800+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb7f800f020 con 0x7fb800105040 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.800+0000 7fb7fe59c700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 0x7fb7e803bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.801+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80006a110 con 0x7fb800105040 2024-04-26T18:15:08.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.801+0000 7fb7fe59c700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 0x7fb7e803bc20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009680 tx=0x7fb7f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:08.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:08.806+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f8025ce0 con 0x7fb800105040 2024-04-26T18:15:09.016 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:08.803+0000 7fe90fd62200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:15:09.016 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:09.016+0000 7fe90fd62200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:15:09.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:09.023+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb80002e900 con 0x7fb7e8039760 2024-04-26T18:15:09.186 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:08.761+0000 7f7648065200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:15:09.186 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:08.870+0000 7f7648065200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:15:09.186 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:09.185+0000 7f7648065200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:15:09.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:09.219+0000 7fe90fd62200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:15:09.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:09.288+0000 7f7648065200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:15:09.818 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:09.819 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:09.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:09.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:09.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:10.301 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:09.873+0000 7fe90fd62200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:15:10.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:10.301+0000 7fe90fd62200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:15:10.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:10.397+0000 7fe90fd62200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:15:10.913 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:10.566+0000 7f7648065200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:15:10.913 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:10.738+0000 7f7648065200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:15:10.913 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:10.912+0000 7f7648065200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:15:10.916 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:10.598+0000 7fe90fd62200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:15:10.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:10 smithi037 ceph-mon[29171]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:15:11.213 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:11.214 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:11.214 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:11.214 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:10 smithi120 ceph-mon[32244]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.132+0000 7fe90fd62200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.223+0000 7fe90fd62200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:15:11.224 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:15:11.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:15:11.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:11.225 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:10 smithi170 ceph-mon[32697]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T18:15:11.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:11.212+0000 7f7648065200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:15:11.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:11.304+0000 7f7648065200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:15:11.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.467+0000 7fe90fd62200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:15:12.002 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:11.505+0000 7f7648065200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:15:12.002 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:11.694+0000 7f7648065200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:15:12.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.597+0000 7fe90fd62200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:15:12.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.693+0000 7fe90fd62200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:15:12.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:11.810+0000 7fe90fd62200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]: dispatch 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]': finished 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:15:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:12.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:12.289+0000 7f7648065200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]: dispatch 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]': finished 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:15:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:12.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:12.116+0000 7fe90fd62200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:15:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]: dispatch 2024-04-26T18:15:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3293575048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c564014-c9c6-4050-8c9e-ab1b1825a7cb"}]': finished 2024-04-26T18:15:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:15:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:13.002 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:12.710+0000 7f7648065200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:15:13.002 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:12.807+0000 7f7648065200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:15:13.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:12.713+0000 7fe90fd62200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:15:13.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:15:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:15:12.930+0000 7fe90fd62200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:15:13.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:13.087+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fb7f8026450 con 0x7fb800105040 2024-04-26T18:15:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/555777089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: Standby manager daemon z started 2024-04-26T18:15:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:15:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:15:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:13 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:13.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:13.009+0000 7f7648065200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:15:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/555777089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: Standby manager daemon z started 2024-04-26T18:15:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:15:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:15:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/555777089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: Standby manager daemon z started 2024-04-26T18:15:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:15:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:15:13.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:13 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/2861301293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:13.857 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:13.525+0000 7f7648065200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:15:13.857 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:13.615+0000 7f7648065200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:15:14.200 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-mon[32244]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-mon[32244]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:13.856+0000 7f7648065200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:13.990+0000 7f7648065200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:14.085+0000 7f7648065200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:15:14.201 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:14.200+0000 7f7648065200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:15:14.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:14 smithi037 ceph-mon[29171]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:14.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:14 smithi037 ceph-mon[29171]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-26T18:15:14.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:15:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:14 smithi170 ceph-mon[32697]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:14 smithi170 ceph-mon[32697]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-26T18:15:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:15:14.752 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:14.498+0000 7f7648065200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:15:15.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:15.094+0000 7f7648065200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:15:15.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:15:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:15:15.307+0000 7f7648065200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:15:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:16.100+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mon.2 v2:172.21.15.120:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb7f8011390 con 0x7fb800105040 2024-04-26T18:15:16.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:16.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: Standby manager daemon y started 2024-04-26T18:15:16.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:15:16.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:16.462 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:15:16.462 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:16 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: Standby manager daemon y started 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:15:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:16 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: Standby manager daemon y started 2024-04-26T18:15:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:15:16.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:15:16.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:15:16.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:16 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/356305952' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:15:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:17 smithi037 ceph-mon[29171]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-26T18:15:17.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:15:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:17 smithi120 ceph-mon[32244]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-26T18:15:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:15:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:17 smithi170 ceph-mon[32697]: mgrmap e15: x(active, since 79s), standbys: y, z 2024-04-26T18:15:17.718 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:15:18.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:18.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:18.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:18.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:18.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:18.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:20.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:20 smithi037 ceph-mon[29171]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:20.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:15:20.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:20 smithi120 ceph-mon[32244]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:20.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:15:20.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:20.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:20 smithi170 ceph-mon[32697]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:20.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:15:20.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:21.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:21 smithi037 ceph-mon[29171]: Deploying daemon osd.0 on smithi037 2024-04-26T18:15:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:21 smithi120 ceph-mon[32244]: Deploying daemon osd.0 on smithi037 2024-04-26T18:15:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:21 smithi170 ceph-mon[32697]: Deploying daemon osd.0 on smithi037 2024-04-26T18:15:22.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:22 smithi037 ceph-mon[29171]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:22.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:22 smithi120 ceph-mon[32244]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:22.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:22 smithi170 ceph-mon[32697]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:24.475 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:24 smithi037 ceph-mon[29171]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:24.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:24 smithi120 ceph-mon[32244]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:24 smithi170 ceph-mon[32697]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:26 smithi037 ceph-mon[29171]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:26.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:26 smithi120 ceph-mon[32244]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:26 smithi170 ceph-mon[32697]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:26.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:28.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:28.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:28.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:28.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:28.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:28.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:28.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:28.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:29.085 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-04-26T18:15:29.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.085+0000 7fb7ef7fe700 1 -- 172.21.15.37:0/1280750126 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb80002e900 con 0x7fb7e8039760 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 msgr2=0x7fb7e803bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 0x7fb7e803bc20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009680 tx=0x7fb7f0009260 comp rx=0 tx=0).stop 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 msgr2=0x7fb800075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800b770 tx=0x7fb7f8004ce0 comp rx=0 tx=0).stop 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 shutdown_connections 2024-04-26T18:15:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb7e8039760 0x7fb7e803bc20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb800105040 0x7fb800075d50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8001034a0 0x7fb8000771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 --2- 172.21.15.37:0/1280750126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8001046a0 0x7fb800075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 >> 172.21.15.37:0/1280750126 conn(0x7fb8000fea40 msgr2=0x7fb8001079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 shutdown_connections 2024-04-26T18:15:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:15:29.087+0000 7fb80495a700 1 -- 172.21.15.37:0/1280750126 wait complete. 2024-04-26T18:15:29.697 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T18:15:29.699 INFO:tasks.cephadm:Deploying osd.1 on smithi120 with /dev/vg_nvme/lv_4... 2024-04-26T18:15:29.699 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:15:29.941 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:15:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:30 smithi037 ceph-mon[29171]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:30 smithi037 ceph-mon[29171]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:15:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:30 smithi170 ceph-mon[32697]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:30 smithi170 ceph-mon[32697]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:15:30.329 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:30 smithi120 ceph-mon[32244]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:30.329 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.329 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:30.329 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:30.329 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:30 smithi120 ceph-mon[32244]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:15:31.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:31 smithi037 ceph-mon[29171]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:15:31.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:31 smithi037 ceph-mon[29171]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:15:31.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:31 smithi037 ceph-mon[29171]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:15:31.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:31 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:31.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:31 smithi120 ceph-mon[32244]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:15:31.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:31 smithi120 ceph-mon[32244]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:15:31.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:31 smithi120 ceph-mon[32244]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:15:31.345 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:31 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:31 smithi170 ceph-mon[32697]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:15:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:31 smithi170 ceph-mon[32697]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:15:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:31 smithi170 ceph-mon[32697]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:15:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:31 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:31.709 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:15:31.709 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:15:31.709 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-04-26T18:15:31.709 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-04-26T18:15:31.710 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125092 s, 838 MB/s 2024-04-26T18:15:31.710 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-04-26T18:15:32.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' 2024-04-26T18:15:32.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.213 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:15:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0[41296]: 2024-04-26T18:15:31.954+0000 7fb440d1c700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T18:15:32.213 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:15:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0[41296]: 2024-04-26T18:15:32.023+0000 7fb438b07700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:15:32.343 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' 2024-04-26T18:15:32.344 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.372 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-04-26T18:15:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:15:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:15:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: from='osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654]' entity='osd.0' 2024-04-26T18:15:32.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:32.612 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:15:33.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:15:33.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:15:33.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] boot 2024-04-26T18:15:33.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:15:33.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] boot 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:15:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:15:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:15:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: osd.0 [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] boot 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:15:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:15:34.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 -- 172.21.15.120:0/3403855427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01022a0 msgr2=0x7f4ee01026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:34.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/3403855427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01022a0 0x7f4ee01026c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00099c0 tx=0x7f4ed0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.068 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 -- 172.21.15.120:0/3403855427 shutdown_connections 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/3403855427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 0x7f4ee010a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/3403855427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01022a0 0x7f4ee01026c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/3403855427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01034a0 0x7f4ee0103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 -- 172.21.15.120:0/3403855427 >> 172.21.15.120:0/3403855427 conn(0x7f4ee00fd840 msgr2=0x7f4ee00ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 -- 172.21.15.120:0/3403855427 shutdown_connections 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.065+0000 7f4ee49fe700 1 -- 172.21.15.120:0/3403855427 wait complete. 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.066+0000 7f4ee49fe700 1 Processor -- start 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.066+0000 7f4ee49fe700 1 -- start start 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01022a0 0x7f4ee006c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 0x7f4ee01a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010de60 con 0x7f4ee0103e40 2024-04-26T18:15:34.069 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010dce0 con 0x7f4ee01034a0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010dfe0 con 0x7f4ee01022a0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56906/0 (socket says 172.21.15.120:56906) 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 -- 172.21.15.120:0/2292641051 learned_addr learned my addr 172.21.15.120:0/2292641051 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4edf59e700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 0x7f4ee01a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4eded9d700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01022a0 0x7f4ee006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 -- 172.21.15.120:0/2292641051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01022a0 msgr2=0x7f4ee006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01022a0 0x7f4ee006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 -- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 msgr2=0x7f4ee01a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 0x7f4ee01a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 -- 172.21.15.120:0/2292641051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee01a1230 con 0x7f4ee01034a0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ede59c700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b530 tx=0x7f4ed400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4015020 con 0x7f4ee01034a0 2024-04-26T18:15:34.070 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee01a1520 con 0x7f4ee01034a0 2024-04-26T18:15:34.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ed4011ce0 con 0x7f4ee01034a0 2024-04-26T18:15:34.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4010e40 con 0x7f4ee01034a0 2024-04-26T18:15:34.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.067+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee01a1a90 con 0x7f4ee01034a0 2024-04-26T18:15:34.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.068+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee006a110 con 0x7f4ee01034a0 2024-04-26T18:15:34.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.069+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f4ed4018020 con 0x7f4ee01034a0 2024-04-26T18:15:34.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.070+0000 7f4ec7fff700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:34.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.070+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f4ed40ce770 con 0x7f4ee01034a0 2024-04-26T18:15:34.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.070+0000 7f4eded9d700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:34.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.071+0000 7f4eded9d700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b580 tx=0x7f4ed0005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:34.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.074+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ed401f020 con 0x7f4ee01034a0 2024-04-26T18:15:34.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:34 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi037 2024-04-26T18:15:34.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:34 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-26T18:15:34.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:34 smithi120 ceph-mon[32244]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:34.280 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:34.280+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4ee0063860 con 0x7f4ec80a3bb0 2024-04-26T18:15:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:34 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi037 2024-04-26T18:15:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:34 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-26T18:15:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:34 smithi037 ceph-mon[29171]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:34 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi037 2024-04-26T18:15:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:34 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-26T18:15:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:34 smithi170 ceph-mon[32697]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:15:35.199 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:35 smithi120 ceph-mon[32244]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:15:35.199 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:35 smithi120 ceph-mon[32244]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:35.199 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:35 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:35.199 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:35 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:35.199 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:35 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:35 smithi037 ceph-mon[29171]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:15:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:35 smithi037 ceph-mon[29171]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:35 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:35 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:35 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:35 smithi170 ceph-mon[32697]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:15:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:35 smithi170 ceph-mon[32697]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:15:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:35 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:35 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:35 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:36.189 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:36 smithi120 ceph-mon[32244]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:36.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:36 smithi170 ceph-mon[32697]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:36.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:36 smithi037 ceph-mon[29171]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:37.403 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:37 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/99750566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.403 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:37 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.404 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:37 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]': finished 2024-04-26T18:15:37.404 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:37 smithi120 ceph-mon[32244]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:15:37.404 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:37 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/99750566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:37 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:37 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]': finished 2024-04-26T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:37 smithi037 ceph-mon[29171]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:15:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:37 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/99750566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:37 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]: dispatch 2024-04-26T18:15:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:37 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d22495-dc0e-4795-9b4c-f9f1d6c4546a"}]': finished 2024-04-26T18:15:37.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:37 smithi170 ceph-mon[32697]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:15:37.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:38.413 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:38 smithi120 ceph-mon[32244]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:38.414 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:38 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/1173104783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:38.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:38 smithi037 ceph-mon[29171]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:38.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:38 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/1173104783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:38.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:38 smithi170 ceph-mon[32697]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:38.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:38 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/1173104783' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:15:40.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:40 smithi037 ceph-mon[29171]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:40.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:40 smithi120 ceph-mon[32244]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:40.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:40 smithi170 ceph-mon[32697]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:42.247 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:42 smithi120 ceph-mon[32244]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:42.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:42 smithi037 ceph-mon[29171]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:42 smithi170 ceph-mon[32697]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:44.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:44 smithi120 ceph-mon[32244]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:44.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:44 smithi037 ceph-mon[29171]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:44 smithi170 ceph-mon[32697]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:45.439 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:15:45.439 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:45.439 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:45 smithi120 ceph-mon[32244]: Deploying daemon osd.1 on smithi120 2024-04-26T18:15:45.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:15:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:45 smithi037 ceph-mon[29171]: Deploying daemon osd.1 on smithi120 2024-04-26T18:15:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:15:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:45 smithi170 ceph-mon[32697]: Deploying daemon osd.1 on smithi120 2024-04-26T18:15:46.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:46 smithi037 ceph-mon[29171]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:46.388 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:46 smithi120 ceph-mon[32244]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:46 smithi170 ceph-mon[32697]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:48.453 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:48 smithi120 ceph-mon[32244]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:48.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:48 smithi037 ceph-mon[29171]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:48 smithi170 ceph-mon[32697]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:50.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:50 smithi120 ceph-mon[32244]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:50 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:50 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:50 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:50.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:50 smithi037 ceph-mon[29171]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:50.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:50 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:50 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:50 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:50 smithi170 ceph-mon[32697]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:50 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:50 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:50 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:52.481 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:52.481 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.481 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.482 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:52.482 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:52.482 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:52.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:52.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:52.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:52.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:52.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:53.152 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.152+0000 7f4ec7fff700 1 -- 172.21.15.120:0/2292641051 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4ee0063860 con 0x7f4ec80a3bb0 2024-04-26T18:15:53.152 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 1 on host 'smithi120' 2024-04-26T18:15:53.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 msgr2=0x7f4ec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:53.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b580 tx=0x7f4ed0005c00 comp rx=0 tx=0).stop 2024-04-26T18:15:53.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 msgr2=0x7f4ee019e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b530 tx=0x7f4ed400b840 comp rx=0 tx=0).stop 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 shutdown_connections 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4ec80a3bb0 0x7f4ec80a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee0103e40 0x7f4ee01a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ee01022a0 0x7f4ee006c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 --2- 172.21.15.120:0/2292641051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ee01034a0 0x7f4ee019e2d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 >> 172.21.15.120:0/2292641051 conn(0x7f4ee00fd840 msgr2=0x7f4ee0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 shutdown_connections 2024-04-26T18:15:53.155 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:15:53.154+0000 7f4ee49fe700 1 -- 172.21.15.120:0/2292641051 wait complete. 2024-04-26T18:15:53.853 DEBUG:teuthology.orchestra.run.smithi120:osd.1> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T18:15:53.855 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2024-04-26T18:15:53.856 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:15:54.098 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:15:54.220 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:54.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:54.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:54.221 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:54 smithi170 ceph-mon[32697]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:54.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:54 smithi037 ceph-mon[29171]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:54 smithi120 ceph-mon[32244]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:15:55.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:55 smithi170 ceph-mon[32697]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:15:55.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:55 smithi170 ceph-mon[32697]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:55 smithi170 ceph-mon[32697]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:15:55.320 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:55.321 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:55 smithi170 ceph-mon[32697]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:55 smithi037 ceph-mon[29171]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:15:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:55 smithi037 ceph-mon[29171]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:55 smithi037 ceph-mon[29171]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:15:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:55 smithi037 ceph-mon[29171]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-mon[32244]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:15:55.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-mon[32244]: from='osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-mon[32244]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:15:55.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:55.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-mon[32244]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:15:55.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1[36846]: 2024-04-26T18:15:55.139+0000 7f5fa8e2c700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T18:15:55.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:15:55 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1[36846]: 2024-04-26T18:15:55.144+0000 7f5fa4c1f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:15:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:15:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:15:55.874 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-26T18:15:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-26T18:15:55.874 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00873151 s, 1.2 GB/s 2024-04-26T18:15:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-26T18:15:56.218 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:56 smithi120 ceph-mon[32244]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:56.218 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:56 smithi120 ceph-mon[32244]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:15:56.218 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:56 smithi120 ceph-mon[32244]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:15:56.218 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.218 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:56 smithi170 ceph-mon[32697]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:56.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:56 smithi170 ceph-mon[32697]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:15:56.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:56 smithi170 ceph-mon[32697]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:15:56.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-04-26T18:15:56.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:56 smithi037 ceph-mon[29171]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:56.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:56 smithi037 ceph-mon[29171]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:15:56.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:56 smithi037 ceph-mon[29171]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:15:56.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:56.694 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:15:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] boot 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] boot 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:57.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:57.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:57.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: osd.1 [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] boot 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- 172.21.15.170:0/1667595953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103580 msgr2=0x7f6a5c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1667595953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103580 0x7f6a5c1039e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009990 tx=0x7f6a48009ca0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- 172.21.15.170:0/1667595953 shutdown_connections 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1667595953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c103f20 0x7f6a5c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1667595953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c102380 0x7f6a5c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1667595953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103580 0x7f6a5c1039e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- 172.21.15.170:0/1667595953 >> 172.21.15.170:0/1667595953 conn(0x7f6a5c0fd920 msgr2=0x7f6a5c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- 172.21.15.170:0/1667595953 shutdown_connections 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- 172.21.15.170:0/1667595953 wait complete. 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 Processor -- start 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- start start 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 0x7f6a5c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103580 0x7f6a5c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10df60 con 0x7f6a5c103580 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10dde0 con 0x7f6a5c102380 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a62d62700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10e0e0 con 0x7f6a5c103f20 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.334+0000 7f6a612ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:58.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51462/0 (socket says 172.21.15.170:51462) 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 -- 172.21.15.170:0/1367197353 learned_addr learned my addr 172.21.15.170:0/1367197353 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a5bfff700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103580 0x7f6a5c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 -- 172.21.15.170:0/1367197353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 msgr2=0x7f6a5c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a60afe700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 0x7f6a5c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 0x7f6a5c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 -- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103580 msgr2=0x7f6a5c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103580 0x7f6a5c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 -- 172.21.15.170:0/1367197353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c1a13d0 con 0x7f6a5c103f20 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a60afe700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 0x7f6a5c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a612ff700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b9a0 tx=0x7f6a5400bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54019070 con 0x7f6a5c103f20 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c1a16c0 con 0x7f6a5c103f20 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a5400cde0 con 0x7f6a5c103f20 2024-04-26T18:15:58.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007370 con 0x7f6a5c103f20 2024-04-26T18:15:58.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.335+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1a1c80 con 0x7f6a5c103f20 2024-04-26T18:15:58.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.336+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c06a110 con 0x7f6a5c103f20 2024-04-26T18:15:58.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.337+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6a5401c070 con 0x7f6a5c103f20 2024-04-26T18:15:58.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.338+0000 7f6a59ffb700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 0x7f6a4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:58.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.338+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f6a540ce120 con 0x7f6a5c103f20 2024-04-26T18:15:58.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.338+0000 7f6a60afe700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 0x7f6a4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:58.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.339+0000 7f6a60afe700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 0x7f6a4c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c1033e0 tx=0x7f6a50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:58.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.343+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a5409c350 con 0x7f6a5c103f20 2024-04-26T18:15:58.343 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:58 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi120 2024-04-26T18:15:58.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:58 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 17088M 2024-04-26T18:15:58.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:58 smithi170 ceph-mon[32697]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:58.344 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:58 smithi170 ceph-mon[32697]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:15:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:58 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi120 2024-04-26T18:15:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:58 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 17088M 2024-04-26T18:15:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:58 smithi037 ceph-mon[29171]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:58 smithi037 ceph-mon[29171]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:15:58.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:58 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi120 2024-04-26T18:15:58.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:58 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 17088M 2024-04-26T18:15:58.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:58 smithi120 ceph-mon[32244]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:15:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:58 smithi120 ceph-mon[32244]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:15:58.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:15:58.549+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6a5c0ffb10 con 0x7f6a4c0a3bb0 2024-04-26T18:15:59.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:15:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:59.491 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:59.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:59.492 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:15:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:15:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:15:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:15:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:00.356 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:00 smithi170 ceph-mon[32697]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:00.356 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:00 smithi170 ceph-mon[32697]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:00 smithi037 ceph-mon[29171]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:00 smithi037 ceph-mon[29171]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:00.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:00 smithi120 ceph-mon[32244]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:00.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:00 smithi120 ceph-mon[32244]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/328692355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]': finished 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:02 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/3162563326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:02.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:02.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/328692355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]': finished 2024-04-26T18:16:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:16:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:02 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/3162563326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/328692355' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]: dispatch 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b1c3764-35d5-4229-b362-722dbbe07c44"}]': finished 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:16:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:02.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:02 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/3162563326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:04.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:04 smithi037 ceph-mon[29171]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:04 smithi120 ceph-mon[32244]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:04.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:04 smithi170 ceph-mon[32697]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:06.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:06 smithi170 ceph-mon[32697]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:06.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:06 smithi037 ceph-mon[29171]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:06.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:06 smithi120 ceph-mon[32244]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:08.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:08 smithi170 ceph-mon[32697]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:08.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:08 smithi037 ceph-mon[29171]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:08.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:08 smithi120 ceph-mon[32244]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:09.209 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:16:09.210 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:09.210 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:09 smithi170 ceph-mon[32697]: Deploying daemon osd.2 on smithi170 2024-04-26T18:16:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:16:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:09 smithi037 ceph-mon[29171]: Deploying daemon osd.2 on smithi170 2024-04-26T18:16:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:16:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:09.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:09 smithi120 ceph-mon[32244]: Deploying daemon osd.2 on smithi170 2024-04-26T18:16:10.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:10 smithi037 ceph-mon[29171]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:10.494 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:10 smithi120 ceph-mon[32244]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:10.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:10 smithi170 ceph-mon[32697]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:12.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:12 smithi037 ceph-mon[29171]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:12 smithi120 ceph-mon[32244]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:12 smithi170 ceph-mon[32697]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:14 smithi170 ceph-mon[32697]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:14.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:14 smithi037 ceph-mon[29171]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:14.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:14.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:14 smithi120 ceph-mon[32244]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:14.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:14.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:16.670 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:16 smithi170 ceph-mon[32697]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:16.721 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-04-26T18:16:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.721+0000 7f6a59ffb700 1 -- 172.21.15.170:0/1367197353 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a5c0ffb10 con 0x7f6a4c0a3bb0 2024-04-26T18:16:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 msgr2=0x7f6a4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 0x7f6a4c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c1033e0 tx=0x7f6a50008040 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 msgr2=0x7f6a5c1a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b9a0 tx=0x7f6a5400bcb0 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 shutdown_connections 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6a4c0a3bb0 0x7f6a4c0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a5c103f20 0x7f6a5c1a0d30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a5c102380 0x7f6a5c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 --2- 172.21.15.170:0/1367197353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103580 0x7f6a5c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 >> 172.21.15.170:0/1367197353 conn(0x7f6a5c0fd920 msgr2=0x7f6a5c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 shutdown_connections 2024-04-26T18:16:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:16:16.723+0000 7f6a62d62700 1 -- 172.21.15.170:0/1367197353 wait complete. 2024-04-26T18:16:16.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:16 smithi037 ceph-mon[29171]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:16.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:16 smithi120 ceph-mon[32244]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:17.352 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T18:16:17.354 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-26T18:16:17.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:16:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:17.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:17 smithi170 ceph-mon[32697]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:17.611 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:17.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:17 smithi037 ceph-mon[29171]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:17.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:17 smithi120 ceph-mon[32244]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:18.463 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:18 smithi170 ceph-mon[32697]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:18 smithi037 ceph-mon[29171]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:18.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:16:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: from='osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:16:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:18 smithi120 ceph-mon[32244]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:16:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:16:19.530 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:16:19.530 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T18:16:19.531 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T18:16:19.531 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128318 s, 817 MB/s 2024-04-26T18:16:19.531 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T18:16:19.716 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2[37832]: 2024-04-26T18:16:19.416+0000 7f82a5e30700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T18:16:19.717 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2[37832]: 2024-04-26T18:16:19.421+0000 7f829dc1b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:19.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:19 smithi037 ceph-mon[29171]: from='osd.2 ' entity='osd.2' 2024-04-26T18:16:20.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:20.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:16:20.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:16:20.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:19 smithi120 ceph-mon[32244]: from='osd.2 ' entity='osd.2' 2024-04-26T18:16:20.063 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:19 smithi170 ceph-mon[32697]: from='osd.2 ' entity='osd.2' 2024-04-26T18:16:20.316 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi170 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:20.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] boot 2024-04-26T18:16:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:16:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:21.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi170 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] boot 2024-04-26T18:16:21.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:16:21.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:21.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:21.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:16:21.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi170 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: osd.2 [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] boot 2024-04-26T18:16:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:16:21.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:16:21.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 -- 172.21.15.37:0/3408947886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 msgr2=0x7f1854104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 --2- 172.21.15.37:0/3408947886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854104ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f18440099c0 tx=0x7f1844009cd0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 -- 172.21.15.37:0/3408947886 shutdown_connections 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 --2- 172.21.15.37:0/3408947886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 0x7f185410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 --2- 172.21.15.37:0/3408947886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 0x7f1854103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 --2- 172.21.15.37:0/3408947886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854104ad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 -- 172.21.15.37:0/3408947886 >> 172.21.15.37:0/3408947886 conn(0x7f18540fea10 msgr2=0x7f1854100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 -- 172.21.15.37:0/3408947886 shutdown_connections 2024-04-26T18:16:21.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.982+0000 7f185b843700 1 -- 172.21.15.37:0/3408947886 wait complete. 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 Processor -- start 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 -- start start 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 0x7f1854199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 0x7f18541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f185410eef0 con 0x7f1854104670 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f185410ed70 con 0x7f1854105010 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f185b843700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f185410f070 con 0x7f1854103470 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:21.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53130/0 (socket says 172.21.15.37:53130) 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 -- 172.21.15.37:0/3082602958 learned_addr learned my addr 172.21.15.37:0/3082602958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1859de0700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 0x7f18541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f18595df700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 0x7f1854199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 -- 172.21.15.37:0/3082602958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 msgr2=0x7f1854199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 0x7f1854199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 -- 172.21.15.37:0/3082602958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 msgr2=0x7f18541a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 0x7f18541a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.983+0000 7f1858dde700 1 -- 172.21.15.37:0/3082602958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541a1190 con 0x7f1854104670 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f1858dde700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1844005910 tx=0x7f1844005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844027070 con 0x7f1854104670 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440040a0 con 0x7f1854104670 2024-04-26T18:16:21.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541a1420 con 0x7f1854104670 2024-04-26T18:16:21.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844022ce0 con 0x7f1854104670 2024-04-26T18:16:21.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.984+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541a1900 con 0x7f1854104670 2024-04-26T18:16:21.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.985+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185406a110 con 0x7f1854104670 2024-04-26T18:16:21.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.986+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f184402a070 con 0x7f1854104670 2024-04-26T18:16:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.987+0000 7f184a7fc700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 0x7f18400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.987+0000 7f18595df700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 0x7f18400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.987+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f18440cfed0 con 0x7f1854104670 2024-04-26T18:16:21.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.987+0000 7f18595df700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 0x7f18400a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f18541044d0 tx=0x7f1850005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:21.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:21.991+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1844013070 con 0x7f1854104670 2024-04-26T18:16:22.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:22.208+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1854063920 con 0x7f18400a3bb0 2024-04-26T18:16:22.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:22.694 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:22.695 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.695 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41901]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.695 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41901]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:22.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:22.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:16:22.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:22.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:22.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:22.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:22.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:16:22.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:16:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:16:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:22.753 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40953]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:22.753 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40953]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.753 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40953]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.753 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40953]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.754 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40956]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:22.754 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.754 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40956]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.754 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40956]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.868 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45638]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45638]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45638]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45638]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45641]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45641]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.869 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45641]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:22.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45685]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45685]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45701]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:22.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:22 smithi037 sudo[45701]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.075 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:23.075 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:23.075 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41904]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:23.075 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41904]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41907]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41907]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41910]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:23.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:22 smithi170 sudo[41910]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40959]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40959]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40959]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40962]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:16:23.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:22 smithi120 sudo[40962]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:23 smithi037 ceph-mon[29171]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:23.614+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1844004250 con 0x7f1854104670 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.755 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:23 smithi120 ceph-mon[32244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:16:23.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:23 smithi170 ceph-mon[32697]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:16:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:24 smithi037 ceph-mon[29171]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:24 smithi037 ceph-mon[29171]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:16:24.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:24 smithi037 ceph-mon[29171]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T18:16:24.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:24 smithi120 ceph-mon[32244]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:24.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:24 smithi120 ceph-mon[32244]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:16:24.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:24 smithi120 ceph-mon[32244]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T18:16:24.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:24 smithi170 ceph-mon[32697]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:24.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:24 smithi170 ceph-mon[32697]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:16:24.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:24 smithi170 ceph-mon[32697]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-26T18:16:25.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:25 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]: dispatch 2024-04-26T18:16:25.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:25 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]': finished 2024-04-26T18:16:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:25 smithi037 ceph-mon[29171]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:16:25.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:25.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:25 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]: dispatch 2024-04-26T18:16:25.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:25 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]': finished 2024-04-26T18:16:25.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:25 smithi120 ceph-mon[32244]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:16:25.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:25.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:25 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]: dispatch 2024-04-26T18:16:25.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:25 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/350530633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26ddcdf7-9137-4384-9228-8537b2009989"}]': finished 2024-04-26T18:16:25.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:25 smithi170 ceph-mon[32697]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:16:25.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:26.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:26 smithi037 ceph-mon[29171]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:26 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4064182582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:26.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:26 smithi120 ceph-mon[32244]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:26 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4064182582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:26 smithi170 ceph-mon[32697]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:26.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:26 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4064182582' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:28.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:28 smithi037 ceph-mon[29171]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:28.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:28 smithi120 ceph-mon[32244]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:28.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:28 smithi170 ceph-mon[32697]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:30.634 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:30 smithi037 ceph-mon[29171]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:30 smithi120 ceph-mon[32244]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:30.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:30 smithi170 ceph-mon[32697]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:32.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:32 smithi037 ceph-mon[29171]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:32.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:32 smithi120 ceph-mon[32244]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:32.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:32 smithi170 ceph-mon[32697]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:33.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:16:33.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:33.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:16:33.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:33.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:16:33.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:34.596 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:34 smithi037 ceph-mon[29171]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:34.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:34 smithi037 ceph-mon[29171]: Deploying daemon osd.3 on smithi037 2024-04-26T18:16:34.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:34 smithi120 ceph-mon[32244]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:34 smithi120 ceph-mon[32244]: Deploying daemon osd.3 on smithi037 2024-04-26T18:16:34.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:34 smithi170 ceph-mon[32697]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:34.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:34 smithi170 ceph-mon[32697]: Deploying daemon osd.3 on smithi037 2024-04-26T18:16:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:36 smithi037 ceph-mon[29171]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:36.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:36 smithi120 ceph-mon[32244]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:36 smithi170 ceph-mon[32697]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:37.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:37 smithi037 ceph-mon[29171]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:37.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:37 smithi120 ceph-mon[32244]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:37.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:37 smithi170 ceph-mon[32697]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:39.555 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.555 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.555 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:39.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:40.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:40 smithi170 ceph-mon[32697]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:40.700 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:40 smithi037 ceph-mon[29171]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:40.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:40 smithi120 ceph-mon[32244]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:41.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.971+0000 7f184a7fc700 1 -- 172.21.15.37:0/3082602958 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1854063920 con 0x7f18400a3bb0 2024-04-26T18:16:41.970 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 msgr2=0x7f18400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 0x7f18400a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f18541044d0 tx=0x7f1850005d70 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 msgr2=0x7f1854199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1844005910 tx=0x7f1844005b00 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 shutdown_connections 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18400a3bb0 0x7f18400a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1854105010 0x7f18541a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1854103470 0x7f1854199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 --2- 172.21.15.37:0/3082602958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1854104670 0x7f1854199a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 >> 172.21.15.37:0/3082602958 conn(0x7f18540fea10 msgr2=0x7f1854107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 shutdown_connections 2024-04-26T18:16:41.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:16:41.973+0000 7f185b843700 1 -- 172.21.15.37:0/3082602958 wait complete. 2024-04-26T18:16:42.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:42.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:42.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:42.643 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T18:16:42.646 INFO:tasks.cephadm:Deploying osd.4 on smithi120 with /dev/vg_nvme/lv_3... 2024-04-26T18:16:42.646 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:16:42.889 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:16:43.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:43 smithi037 ceph-mon[29171]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:16:43.319 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:43 smithi120 ceph-mon[32244]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:16:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:43 smithi170 ceph-mon[32697]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:16:44.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:44 smithi120 ceph-mon[32244]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:44.150 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:44 smithi120 ceph-mon[32244]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:16:44.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:44 smithi120 ceph-mon[32244]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:16:44.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:44 smithi120 ceph-mon[32244]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:16:44.151 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:44 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:44.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-mon[29171]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:44.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-mon[29171]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:16:44.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-mon[29171]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:16:44.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-mon[29171]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:16:44.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:44.484 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:16:44.117+0000 7feb98dc9700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T18:16:44.484 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:16:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:16:44.125+0000 7feb94bbc700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:16:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:44 smithi170 ceph-mon[32697]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:44 smithi170 ceph-mon[32697]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:16:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:44 smithi170 ceph-mon[32697]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:16:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:44 smithi170 ceph-mon[32697]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:16:44.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:44 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:44.733 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:16:44.733 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:16:44.733 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-04-26T18:16:44.733 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-04-26T18:16:44.734 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111161 s, 943 MB/s 2024-04-26T18:16:44.734 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-04-26T18:16:45.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] boot 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:16:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] boot 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:16:45.346 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.372 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-04-26T18:16:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: from='osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:16:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:16:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: osd.3 [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] boot 2024-04-26T18:16:45.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:16:45.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:16:45.621 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:16:46.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:16:46.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:16:46.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:46.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:46.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:46 smithi120 ceph-mon[32244]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:46 smithi037 ceph-mon[29171]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:16:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:16:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:16:46.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:46 smithi170 ceph-mon[32697]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- 172.21.15.120:0/1112362927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 msgr2=0x7fa36c10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- 172.21.15.120:0/1112362927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c10ba60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b0a0 tx=0x7fa36400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- 172.21.15.120:0/1112362927 shutdown_connections 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- 172.21.15.120:0/1112362927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c10ba60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- 172.21.15.120:0/1112362927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 0x7fa36c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.110 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- 172.21.15.120:0/1112362927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 0x7fa36c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- 172.21.15.120:0/1112362927 >> 172.21.15.120:0/1112362927 conn(0x7fa36c0fd090 msgr2=0x7fa36c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- 172.21.15.120:0/1112362927 shutdown_connections 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- 172.21.15.120:0/1112362927 wait complete. 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 Processor -- start 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- start start 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 0x7fa36c104170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 0x7fa36c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:47.111 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10efb0 con 0x7fa36c1014f0 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10ee30 con 0x7fa36c101e90 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.108+0000 7fa372bbd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c10f130 con 0x7fa36c104a70 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38912/0 (socket says 172.21.15.120:38912) 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 -- 172.21.15.120:0/3407689898 learned_addr learned my addr 172.21.15.120:0/3407689898 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa37115a700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 0x7fa36c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa370959700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 0x7fa36c104170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 -- 172.21.15.120:0/3407689898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 msgr2=0x7fa36c102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 0x7fa36c102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 -- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 msgr2=0x7fa36c104170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:47.112 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 0x7fa36c104170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 -- 172.21.15.120:0/3407689898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c1035b0 con 0x7fa36c101e90 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa36bfff700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c920 tx=0x7fa35800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358007b00 con 0x7fa36c101e90 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c1a3730 con 0x7fa36c101e90 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa358004ce0 con 0x7fa36c101e90 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358005550 con 0x7fa36c101e90 2024-04-26T18:16:47.113 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.109+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c1a3cc0 con 0x7fa36c101e90 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.110+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36c06a110 con 0x7fa36c101e90 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.111+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa358007ca0 con 0x7fa36c101e90 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.112+0000 7fa369ffb700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 0x7fa35c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.112+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fa3580cf100 con 0x7fa36c101e90 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.112+0000 7fa370959700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 0x7fa35c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:47.114 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.112+0000 7fa370959700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 0x7fa35c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa360006fd0 tx=0x7fa360008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:47.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.117+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa358014030 con 0x7fa36c101e90 2024-04-26T18:16:47.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:47 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi037 2024-04-26T18:16:47.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:47 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-26T18:16:47.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:47 smithi120 ceph-mon[32244]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:16:47.323 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:16:47.323+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa36c063860 con 0x7fa35c0a3c40 2024-04-26T18:16:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:47 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi037 2024-04-26T18:16:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:47 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-26T18:16:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:47 smithi037 ceph-mon[29171]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:16:47.552 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:47 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi037 2024-04-26T18:16:47.552 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:47 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-26T18:16:47.552 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:47 smithi170 ceph-mon[32697]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:16:48.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:48 smithi120 ceph-mon[32244]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:16:48.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:48 smithi120 ceph-mon[32244]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:48.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:48 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:48.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:48 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:48.201 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:48 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:48 smithi037 ceph-mon[29171]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:16:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:48 smithi037 ceph-mon[29171]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:48 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:48 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:48.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:48 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:48 smithi170 ceph-mon[32697]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:16:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:48 smithi170 ceph-mon[32697]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:48 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:16:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:48 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:16:48.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:48 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:49.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:49 smithi170 ceph-mon[32697]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:16:49.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:49 smithi037 ceph-mon[29171]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:16:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:49 smithi120 ceph-mon[32244]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/812313501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]': finished 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:16:50.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:50 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:16:50.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:50.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T18:16:50.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:16:50.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/812313501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]': finished 2024-04-26T18:16:50.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:16:50.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:50 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/812313501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]: dispatch 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "633a4362-a88a-4725-b10e-6c893bacd0ee"}]': finished 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:16:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:50 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:16:51.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:51 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/1117917298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:51.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:51 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/1117917298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:51 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/1117917298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:16:52.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:52 smithi037 ceph-mon[29171]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:52.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:52 smithi120 ceph-mon[32244]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:52 smithi170 ceph-mon[32697]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-26T18:16:53.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:53 smithi037 ceph-mon[29171]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T18:16:53.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:53 smithi037 ceph-mon[29171]: Cluster is now healthy 2024-04-26T18:16:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:53 smithi120 ceph-mon[32244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T18:16:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:53 smithi120 ceph-mon[32244]: Cluster is now healthy 2024-04-26T18:16:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:53 smithi170 ceph-mon[32697]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-26T18:16:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:53 smithi170 ceph-mon[32697]: Cluster is now healthy 2024-04-26T18:16:54.458 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:54 smithi120 ceph-mon[32244]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:16:54.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:54 smithi037 ceph-mon[29171]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:16:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:54 smithi170 ceph-mon[32697]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:16:56.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:56 smithi120 ceph-mon[32244]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T18:16:56.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:56 smithi037 ceph-mon[29171]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T18:16:56.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:56 smithi170 ceph-mon[32697]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T18:16:58.230 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:58 smithi120 ceph-mon[32244]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:16:58.231 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:16:58.231 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:58.231 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:16:58 smithi120 ceph-mon[32244]: Deploying daemon osd.4 on smithi120 2024-04-26T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:58 smithi037 ceph-mon[29171]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:16:58 smithi037 ceph-mon[29171]: Deploying daemon osd.4 on smithi120 2024-04-26T18:16:58.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:58 smithi170 ceph-mon[32697]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:16:58.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:16:58.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:58.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:16:58 smithi170 ceph-mon[32697]: Deploying daemon osd.4 on smithi120 2024-04-26T18:17:00.436 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:00 smithi120 ceph-mon[32244]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T18:17:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:00 smithi037 ceph-mon[29171]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T18:17:00.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:00 smithi170 ceph-mon[32697]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T18:17:02.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:02 smithi037 ceph-mon[29171]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-26T18:17:02.487 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:02 smithi120 ceph-mon[32244]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-26T18:17:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:02 smithi170 ceph-mon[32697]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-26T18:17:03.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:03.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:03.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:03.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:03 smithi170 ceph-mon[32697]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:17:03.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:03.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:03.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:03.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:03 smithi037 ceph-mon[29171]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:17:04.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:04.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:04.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:04.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:03 smithi120 ceph-mon[32244]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:17:06.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:06.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:06.103 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:06.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.145 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.144+0000 7fa369ffb700 1 -- 172.21.15.120:0/3407689898 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa36c063860 con 0x7fa35c0a3c40 2024-04-26T18:17:06.145 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 4 on host 'smithi120' 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 msgr2=0x7fa35c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 0x7fa35c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa360006fd0 tx=0x7fa360008040 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 msgr2=0x7fa36c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c920 tx=0x7fa35800cc30 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 shutdown_connections 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fa35c0a3c40 0x7fa35c0a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa36c104a70 0x7fa36c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa36c1014f0 0x7fa36c104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 --2- 172.21.15.120:0/3407689898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa36c101e90 0x7fa36c1027c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 >> 172.21.15.120:0/3407689898 conn(0x7fa36c0fd090 msgr2=0x7fa36c107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.147+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 shutdown_connections 2024-04-26T18:17:06.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:17:06.148+0000 7fa372bbd700 1 -- 172.21.15.120:0/3407689898 wait complete. 2024-04-26T18:17:06.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:06.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:06.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:06.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:06.703 DEBUG:teuthology.orchestra.run.smithi120:osd.4> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T18:17:06.705 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-04-26T18:17:06.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:17:06.951 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:17:07.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:07 smithi120 ceph-mon[32244]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:07.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:07 smithi120 ceph-mon[32244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:07.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:07 smithi170 ceph-mon[32697]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:07.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:07 smithi170 ceph-mon[32697]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:07.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:07 smithi037 ceph-mon[29171]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:07 smithi037 ceph-mon[29171]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:08.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:08 smithi170 ceph-mon[32697]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:08.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:17:08.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:17:08.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:08.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:08 smithi037 ceph-mon[29171]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: from='osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:08 smithi120 ceph-mon[32244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:17:08.772 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:17:08.773 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:17:08.773 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-26T18:17:08.773 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-26T18:17:08.773 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00881411 s, 1.2 GB/s 2024-04-26T18:17:08.773 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-26T18:17:09.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:09 smithi170 ceph-mon[32697]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:17:09.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:09 smithi170 ceph-mon[32697]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:17:09.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.370 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:09 smithi170 ceph-mon[32697]: from='osd.4 ' entity='osd.4' 2024-04-26T18:17:09.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-04-26T18:17:09.375 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-mon[32244]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:17:09.375 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-mon[32244]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:17:09.375 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.376 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.376 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-mon[32244]: from='osd.4 ' entity='osd.4' 2024-04-26T18:17:09.376 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:17:09.057+0000 7f83c1525700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T18:17:09.376 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:17:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:17:09.144+0000 7f83b9310700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:17:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:09 smithi037 ceph-mon[29171]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:17:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:09 smithi037 ceph-mon[29171]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:17:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:09 smithi037 ceph-mon[29171]: from='osd.4 ' entity='osd.4' 2024-04-26T18:17:09.615 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:17:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:17:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:17:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] boot 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:17:10.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:10.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:17:10.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:17:10.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:10.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:10.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] boot 2024-04-26T18:17:10.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:17:10.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:17:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:17:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:17:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:10.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:10.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: osd.4 [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] boot 2024-04-26T18:17:10.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:17:10.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:17:11.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 -- 172.21.15.170:0/3715800957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c109190 msgr2=0x7fcd3c10b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 --2- 172.21.15.170:0/3715800957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c109190 0x7fcd3c10b600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b0a0 tx=0x7fcd3400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 -- 172.21.15.170:0/3715800957 shutdown_connections 2024-04-26T18:17:11.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 --2- 172.21.15.170:0/3715800957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c109190 0x7fcd3c10b600 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 --2- 172.21.15.170:0/3715800957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c103f30 0x7fcd3c106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 --2- 172.21.15.170:0/3715800957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c106860 0x7fcd3c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 -- 172.21.15.170:0/3715800957 >> 172.21.15.170:0/3715800957 conn(0x7fcd3c0fd940 msgr2=0x7fcd3c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 -- 172.21.15.170:0/3715800957 shutdown_connections 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.249+0000 7fcd437d3700 1 -- 172.21.15.170:0/3715800957 wait complete. 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 Processor -- start 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 -- start start 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 0x7fcd3c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c109190 0x7fcd3c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10ef00 con 0x7fcd3c109190 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10ed80 con 0x7fcd3c103f30 2024-04-26T18:17:11.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd437d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c10f080 con 0x7fcd3c106860 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52476/0 (socket says 172.21.15.170:52476) 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 -- 172.21.15.170:0/32081831 learned_addr learned my addr 172.21.15.170:0/32081831 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd41d70700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c109190 0x7fcd3c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd4156f700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 0x7fcd3c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 -- 172.21.15.170:0/32081831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 msgr2=0x7fcd3c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 0x7fcd3c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 -- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c109190 msgr2=0x7fcd3c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c109190 0x7fcd3c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd40d6e700 1 -- 172.21.15.170:0/32081831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1a10f0 con 0x7fcd3c106860 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.250+0000 7fcd4156f700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 0x7fcd3c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd40d6e700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00d980 tx=0x7fcd2c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c016070 con 0x7fcd3c106860 2024-04-26T18:17:11.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2c00de70 con 0x7fcd3c106860 2024-04-26T18:17:11.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1a13e0 con 0x7fcd3c106860 2024-04-26T18:17:11.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00f3c0 con 0x7fcd3c106860 2024-04-26T18:17:11.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.251+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1a1920 con 0x7fcd3c106860 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.252+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c06a110 con 0x7fcd3c106860 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.253+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcd2c019070 con 0x7fcd3c106860 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.253+0000 7fcd327fc700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 0x7fcd280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.253+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fcd2c0ceea0 con 0x7fcd3c106860 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.254+0000 7fcd4156f700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 0x7fcd280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.254+0000 7fcd4156f700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 0x7fcd280a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd38005fd0 tx=0x7fcd38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:11.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.257+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd2c09c8f0 con 0x7fcd3c106860 2024-04-26T18:17:11.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:11 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi120 2024-04-26T18:17:11.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:11 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 8544M 2024-04-26T18:17:11.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:11.464+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcd3c063860 con 0x7fcd280a3c40 2024-04-26T18:17:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:11 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi120 2024-04-26T18:17:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:11 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 8544M 2024-04-26T18:17:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:11 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi120 2024-04-26T18:17:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:11 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 8544M 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:12.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:17:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:14.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:14.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/1393455369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]': finished 2024-04-26T18:17:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:17:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/1393455369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]': finished 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:17:14.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:14.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:14.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/1393455369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]: dispatch 2024-04-26T18:17:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "493dab6b-f382-47e3-9a56-480a1328eda1"}]': finished 2024-04-26T18:17:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:17:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:15.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:15 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/1417111668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:15 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/1417111668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:15.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:15 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/1417111668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:16.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:16 smithi037 ceph-mon[29171]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:16.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:16 smithi120 ceph-mon[32244]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:16 smithi170 ceph-mon[32697]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:18.376 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:18 smithi170 ceph-mon[32697]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:18.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:18 smithi037 ceph-mon[29171]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:18.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:18 smithi120 ceph-mon[32244]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:20.236 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:20 smithi170 ceph-mon[32697]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:20.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:20 smithi037 ceph-mon[29171]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:20 smithi120 ceph-mon[32244]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:22.360 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:22 smithi170 ceph-mon[32697]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:22.360 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:17:22.360 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:22.360 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:22 smithi170 ceph-mon[32697]: Deploying daemon osd.5 on smithi170 2024-04-26T18:17:22.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:22 smithi037 ceph-mon[29171]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:17:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:22.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:22 smithi037 ceph-mon[29171]: Deploying daemon osd.5 on smithi170 2024-04-26T18:17:22.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:22 smithi120 ceph-mon[32244]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:17:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:22 smithi120 ceph-mon[32244]: Deploying daemon osd.5 on smithi170 2024-04-26T18:17:24.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:24 smithi170 ceph-mon[32697]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:24.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:24 smithi037 ceph-mon[29171]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:24.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:24 smithi120 ceph-mon[32244]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:26.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:26 smithi170 ceph-mon[32697]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:26.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:26 smithi037 ceph-mon[29171]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:26.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:26 smithi120 ceph-mon[32244]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:27.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:27 smithi037 ceph-mon[29171]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:27.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:27 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:27.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:27 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:27.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:27 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:28.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:27 smithi120 ceph-mon[32244]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:28.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:27 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:28.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:27 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:28.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:27 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:27 smithi170 ceph-mon[32697]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:27 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:27 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:27 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:30.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.383+0000 7fcd327fc700 1 -- 172.21.15.170:0/32081831 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd3c063860 con 0x7fcd280a3c40 2024-04-26T18:17:30.385 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-04-26T18:17:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 msgr2=0x7fcd280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 0x7fcd280a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd38005fd0 tx=0x7fcd38005e30 comp rx=0 tx=0).stop 2024-04-26T18:17:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 msgr2=0x7fcd3c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00d980 tx=0x7fcd2c00dc90 comp rx=0 tx=0).stop 2024-04-26T18:17:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 shutdown_connections 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcd280a3c40 0x7fcd280a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd3c109190 0x7fcd3c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd3c103f30 0x7fcd3c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 --2- 172.21.15.170:0/32081831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd3c106860 0x7fcd3c1999d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 >> 172.21.15.170:0/32081831 conn(0x7fcd3c0fd940 msgr2=0x7fcd3c104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 shutdown_connections 2024-04-26T18:17:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:17:30.386+0000 7fcd437d3700 1 -- 172.21.15.170:0/32081831 wait complete. 2024-04-26T18:17:30.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:30 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:30 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:30.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:30 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:31.067 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T18:17:31.069 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-26T18:17:31.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:17:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:31.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:31 smithi037 ceph-mon[29171]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:31.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:31 smithi037 ceph-mon[29171]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:31.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:31 smithi120 ceph-mon[32244]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:31.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:31 smithi120 ceph-mon[32244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:31.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:31 smithi170 ceph-mon[32697]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:31.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:31 smithi170 ceph-mon[32697]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:32.724 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:32 smithi037 ceph-mon[29171]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:32.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:17:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:17:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:32 smithi120 ceph-mon[32244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:32.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:17:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: from='osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:17:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-mon[32697]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:17:32.826 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:17:32.484+0000 7fa8e1455700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T18:17:32.826 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:17:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:17:32.492+0000 7fa8dd248700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:17:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:33 smithi170 ceph-mon[32697]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:17:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:33 smithi170 ceph-mon[32697]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:17:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:33.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:33.606 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:17:33.606 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:17:33.606 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T18:17:33.607 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T18:17:33.607 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111765 s, 938 MB/s 2024-04-26T18:17:33.607 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T18:17:33.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:33 smithi120 ceph-mon[32244]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:17:33.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:33 smithi120 ceph-mon[32244]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:17:33.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:33.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:33 smithi037 ceph-mon[29171]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:17:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:33 smithi037 ceph-mon[29171]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:17:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:33.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.291 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-26T18:17:34.541 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:34.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:17:34.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:17:34.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:34.664 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] boot 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:34.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] boot 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:34.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:34.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: osd.5 [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] boot 2024-04-26T18:17:34.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:34.827 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:35.658 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:35 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi170 2024-04-26T18:17:35.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:35 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-26T18:17:35.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:35 smithi037 ceph-mon[29171]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:17:35.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:35 smithi037 ceph-mon[29171]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:35.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:35 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi170 2024-04-26T18:17:35.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:35 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-26T18:17:35.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:35 smithi120 ceph-mon[32244]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:17:35.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:35 smithi120 ceph-mon[32244]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:35.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:35 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi170 2024-04-26T18:17:35.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:35 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-26T18:17:35.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:35 smithi170 ceph-mon[32697]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:17:35.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:35 smithi170 ceph-mon[32697]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:36.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 -- 172.21.15.37:0/3756277179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 msgr2=0x7fae0c10a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 --2- 172.21.15.37:0/3756277179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c10a6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae0400b0d0 tx=0x7fae0400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 -- 172.21.15.37:0/3756277179 shutdown_connections 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 --2- 172.21.15.37:0/3756277179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c10a6f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 --2- 172.21.15.37:0/3756277179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1022a0 0x7fae0c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 --2- 172.21.15.37:0/3756277179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1034a0 0x7fae0c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 -- 172.21.15.37:0/3756277179 >> 172.21.15.37:0/3756277179 conn(0x7fae0c0fd840 msgr2=0x7fae0c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 -- 172.21.15.37:0/3756277179 shutdown_connections 2024-04-26T18:17:36.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.146+0000 7fae133a2700 1 -- 172.21.15.37:0/3756277179 wait complete. 2024-04-26T18:17:36.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 Processor -- start 2024-04-26T18:17:36.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 -- start start 2024-04-26T18:17:36.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1022a0 0x7fae0c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1034a0 0x7fae0c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10dd90 con 0x7fae0c103e40 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10dc10 con 0x7fae0c1034a0 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae133a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10df10 con 0x7fae0c1022a0 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57012/0 (socket says 172.21.15.37:57012) 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 -- 172.21.15.37:0/3919865336 learned_addr learned my addr 172.21.15.37:0/3919865336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae11b9f700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1034a0 0x7fae0c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae123a0700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1022a0 0x7fae0c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:36.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 -- 172.21.15.37:0/3919865336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1022a0 msgr2=0x7fae0c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1022a0 0x7fae0c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 -- 172.21.15.37:0/3919865336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1034a0 msgr2=0x7fae0c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1034a0 0x7fae0c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.147+0000 7fae12ba1700 1 -- 172.21.15.37:0/3919865336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c1a10f0 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae12ba1700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae04004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0401f030 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae04003a00 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c1a1380 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae040046a0 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.148+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a1850 con 0x7fae0c103e40 2024-04-26T18:17:36.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.149+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c06a110 con 0x7fae0c103e40 2024-04-26T18:17:36.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.151+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae04003bb0 con 0x7fae0c103e40 2024-04-26T18:17:36.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.151+0000 7fae037fe700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 0x7fadf80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:36.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.151+0000 7fae123a0700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 0x7fadf80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:36.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.151+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fae0400f020 con 0x7fae0c103e40 2024-04-26T18:17:36.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.152+0000 7fae123a0700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 0x7fadf80a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae08005950 tx=0x7fae080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:36.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.155+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae04029020 con 0x7fae0c103e40 2024-04-26T18:17:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:36.373+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fae0c0ff9c0 con 0x7fadf80a3c40 2024-04-26T18:17:36.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:36 smithi037 ceph-mon[29171]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:17:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:36 smithi037 ceph-mon[29171]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:36.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:36 smithi120 ceph-mon[32244]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:17:36.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:36 smithi120 ceph-mon[32244]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:36.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:36.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:36.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:36 smithi170 ceph-mon[32697]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:17:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:36 smithi170 ceph-mon[32697]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:17:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:17:36.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:37 smithi037 ceph-mon[29171]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:17:37.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:37 smithi037 ceph-mon[29171]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:37.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:37 smithi170 ceph-mon[32697]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:17:37.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:37 smithi170 ceph-mon[32697]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:38.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:37 smithi120 ceph-mon[32244]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:17:38.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:37 smithi120 ceph-mon[32244]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:38 smithi037 ceph-mon[29171]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:17:38.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:38 smithi170 ceph-mon[32697]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:17:39.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:38 smithi120 ceph-mon[32244]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:17:39.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-26T18:17:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2304838243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]': finished 2024-04-26T18:17:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T18:17:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2304838243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]': finished 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T18:17:39.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:39.904 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-26T18:17:39.904 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2304838243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.904 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]: dispatch 2024-04-26T18:17:39.904 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "630f5f5f-942b-4a8c-be69-8b1f55491a3d"}]': finished 2024-04-26T18:17:39.904 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T18:17:39.905 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:40.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:40 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1957898896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:40.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:40 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1957898896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:41.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:40 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1957898896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:17:41.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:41 smithi170 ceph-mon[32697]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T18:17:41.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:41 smithi037 ceph-mon[29171]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T18:17:42.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:41 smithi120 ceph-mon[32244]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-26T18:17:43.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:43 smithi037 ceph-mon[29171]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T18:17:44.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:43 smithi120 ceph-mon[32244]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T18:17:44.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:43 smithi170 ceph-mon[32697]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-26T18:17:45.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:45 smithi037 ceph-mon[29171]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:17:46.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:45 smithi120 ceph-mon[32244]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:17:46.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:45 smithi170 ceph-mon[32697]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:17:47.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:47 smithi037 ceph-mon[29171]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:47.785 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:47 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:17:47.785 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:47 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:47.785 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:47 smithi037 ceph-mon[29171]: Deploying daemon osd.6 on smithi037 2024-04-26T18:17:48.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:47 smithi120 ceph-mon[32244]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:48.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:47 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:17:48.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:47 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:48.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:47 smithi120 ceph-mon[32244]: Deploying daemon osd.6 on smithi037 2024-04-26T18:17:48.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:47 smithi170 ceph-mon[32697]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:48.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:47 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:17:48.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:47 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:48.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:47 smithi170 ceph-mon[32697]: Deploying daemon osd.6 on smithi037 2024-04-26T18:17:49.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:49 smithi037 ceph-mon[29171]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:49 smithi120 ceph-mon[32244]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:50.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:49 smithi170 ceph-mon[32697]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:51.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:51 smithi037 ceph-mon[29171]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:52.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:51 smithi120 ceph-mon[32244]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:52.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:51 smithi170 ceph-mon[32697]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:53.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:53 smithi037 ceph-mon[29171]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:53.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:53.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:53.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:54.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:53 smithi120 ceph-mon[32244]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:54.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:54.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:53 smithi170 ceph-mon[32697]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:54.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:54.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:54.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:56.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:56.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:56.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:56.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.602+0000 7fae037fe700 1 -- 172.21.15.37:0/3919865336 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae0c0ff9c0 con 0x7fadf80a3c40 2024-04-26T18:17:56.603 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-04-26T18:17:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 msgr2=0x7fadf80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 0x7fadf80a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae08005950 tx=0x7fae080058e0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 msgr2=0x7fae0c1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae04009fd0 tx=0x7fae04004d50 comp rx=0 tx=0).stop 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 shutdown_connections 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadf80a3c40 0x7fadf80a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c103e40 0x7fae0c1a0a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae0c1022a0 0x7fae0c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 --2- 172.21.15.37:0/3919865336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae0c1034a0 0x7fae0c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 >> 172.21.15.37:0/3919865336 conn(0x7fae0c0fd840 msgr2=0x7fae0c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 shutdown_connections 2024-04-26T18:17:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:17:56.605+0000 7fae133a2700 1 -- 172.21.15.37:0/3919865336 wait complete. 2024-04-26T18:17:56.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:56 smithi037 ceph-mon[29171]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:17:56.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:56.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:57.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:56 smithi120 ceph-mon[32244]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:17:57.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:57.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:57.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:57.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:56 smithi170 ceph-mon[32697]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:17:57.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:57.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:17:57.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:57.299 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T18:17:57.301 INFO:tasks.cephadm:Deploying osd.7 on smithi120 with /dev/vg_nvme/lv_2... 2024-04-26T18:17:57.301 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:17:57.548 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:17:57.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:57 smithi120 ceph-mon[32244]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:57.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:57 smithi120 ceph-mon[32244]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:17:57.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:57 smithi120 ceph-mon[32244]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:17:57.942 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:57 smithi120 ceph-mon[32244]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:17:57.943 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:57.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:57 smithi037 ceph-mon[29171]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:57.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:57 smithi037 ceph-mon[29171]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:17:57.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:57 smithi037 ceph-mon[29171]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:17:57.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:57 smithi037 ceph-mon[29171]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:17:57.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:57 smithi170 ceph-mon[32697]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:58.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:57 smithi170 ceph-mon[32697]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:17:58.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:57 smithi170 ceph-mon[32697]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:17:58.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:57 smithi170 ceph-mon[32697]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:17:58.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' 2024-04-26T18:17:58.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.776 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:17:58.583+0000 7f0568e62700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T18:17:58.776 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:17:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:17:58.715+0000 7f0560c4d700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:17:58.906 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:17:58.907 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:17:58.907 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.907 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:58.907 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' 2024-04-26T18:17:58.907 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:59.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:17:59.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:17:59.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:59.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:59.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: from='osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561]' entity='osd.6' 2024-04-26T18:17:59.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128162 s, 818 MB/s 2024-04-26T18:17:59.555 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-04-26T18:17:59.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:59 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:17:59.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:59 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:17:59.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:59 smithi037 ceph-mon[29171]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:59.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:17:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:17:59.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:59 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:17:59.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:59 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:17:59.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:59 smithi120 ceph-mon[32244]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:17:59.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:17:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:18:00.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:59 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:18:00.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:59 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:18:00.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:59 smithi170 ceph-mon[32697]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:18:00.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:17:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:18:00.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-04-26T18:18:00.424 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:18:00.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] boot 2024-04-26T18:18:00.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T18:18:00.863 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi037 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-26T18:18:00.864 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.865 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:00.865 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:00.865 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:00 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] boot 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi037 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:00.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:00 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:01.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: osd.6 [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] boot 2024-04-26T18:18:01.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T18:18:01.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi037 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:01.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:00 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:01.923 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- 172.21.15.120:0/2042570874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778102370 msgr2=0x7f8778102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- 172.21.15.120:0/2042570874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778102370 0x7f8778102790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f876c0099c0 tx=0x7f876c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- 172.21.15.120:0/2042570874 shutdown_connections 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- 172.21.15.120:0/2042570874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778103f10 0x7f877810a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- 172.21.15.120:0/2042570874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778102370 0x7f8778102790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- 172.21.15.120:0/2042570874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 0x7f87781039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- 172.21.15.120:0/2042570874 >> 172.21.15.120:0/2042570874 conn(0x7f87780fd8f0 msgr2=0x7f87780ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- 172.21.15.120:0/2042570874 shutdown_connections 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- 172.21.15.120:0/2042570874 wait complete. 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 Processor -- start 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- start start 2024-04-26T18:18:01.924 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778102370 0x7f877806c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 0x7f877819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f877810de10 con 0x7f8778103570 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f877810dc90 con 0x7f8778103f10 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.921+0000 7f877efc6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f877810df90 con 0x7f8778102370 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:33502/0 (socket says 172.21.15.120:33502) 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 -- 172.21.15.120:0/2336772853 learned_addr learned my addr 172.21.15.120:0/2336772853 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877cd62700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778102370 0x7f877806c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f8777fff700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 0x7f877819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:01.925 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 -- 172.21.15.120:0/2336772853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778102370 msgr2=0x7f877806c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778102370 0x7f877806c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 -- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 msgr2=0x7f877819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 0x7f877819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 -- 172.21.15.120:0/2336772853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87781a1240 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877d563700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8770007ee0 tx=0x7f877000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770004bb0 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8770004d50 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87781a1530 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.922+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770005920 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.923+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87781a1a20 con 0x7f8778103f10 2024-04-26T18:18:01.926 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.924+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877806a110 con 0x7f8778103f10 2024-04-26T18:18:01.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.925+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f877001a020 con 0x7f8778103f10 2024-04-26T18:18:01.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.925+0000 7f8775ffb700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 0x7f87680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:01.927 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.925+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f87700ced90 con 0x7f8778103f10 2024-04-26T18:18:01.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.926+0000 7f877cd62700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 0x7f87680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:01.928 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.926+0000 7f877cd62700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 0x7f87680a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f876c0038c0 tx=0x7f876c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:01.929 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:01.930+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8770010070 con 0x7f8778103f10 2024-04-26T18:18:02.141 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:02.137+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8778063860 con 0x7f87680a3c40 2024-04-26T18:18:02.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:02 smithi037 ceph-mon[29171]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:18:02.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:02 smithi037 ceph-mon[29171]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:18:02.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:02.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:02.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:02.750 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:02 smithi120 ceph-mon[32244]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:18:02.750 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:02 smithi120 ceph-mon[32244]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:18:02.750 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:02.750 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:02.750 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:02.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:02 smithi170 ceph-mon[32697]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:18:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:02 smithi170 ceph-mon[32697]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:18:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:02.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:03.699 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:03 smithi120 ceph-mon[32244]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:03 smithi037 ceph-mon[29171]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:03.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:03 smithi170 ceph-mon[32697]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:04.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:04 smithi037 ceph-mon[29171]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:04 smithi120 ceph-mon[32244]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:04.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:04 smithi170 ceph-mon[32697]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:05 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/1404077298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:05 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:05 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]': finished 2024-04-26T18:18:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:05 smithi120 ceph-mon[32244]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T18:18:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:05 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:05.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:05 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/1404077298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:05 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:05 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]': finished 2024-04-26T18:18:05.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:05 smithi037 ceph-mon[29171]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T18:18:05.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:05 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:05.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:05 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/1404077298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:05 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]: dispatch 2024-04-26T18:18:05.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:05 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78b9cc68-5517-46de-858e-342ba834c2f8"}]': finished 2024-04-26T18:18:05.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:05 smithi170 ceph-mon[32697]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T18:18:05.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:05 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:06.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:06 smithi170 ceph-mon[32697]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:06.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:06 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/316135283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:06.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:06 smithi037 ceph-mon[29171]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:06.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:06 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/316135283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:06.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:06 smithi120 ceph-mon[32244]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:06.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:06 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/316135283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:08.726 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:08 smithi120 ceph-mon[32244]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:08.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:08 smithi037 ceph-mon[29171]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:08.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:08 smithi170 ceph-mon[32697]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:10.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:10 smithi037 ceph-mon[29171]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:10.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:10 smithi120 ceph-mon[32244]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:10.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:10 smithi170 ceph-mon[32697]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:12.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:12 smithi037 ceph-mon[29171]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:18:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:12.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:12 smithi120 ceph-mon[32244]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:12.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:18:12.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:12.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:12 smithi170 ceph-mon[32697]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:12.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:18:12.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.683 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:13 smithi120 ceph-mon[32244]: Deploying daemon osd.7 on smithi120 2024-04-26T18:18:13.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:13 smithi037 ceph-mon[29171]: Deploying daemon osd.7 on smithi120 2024-04-26T18:18:13.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:13 smithi170 ceph-mon[32697]: Deploying daemon osd.7 on smithi120 2024-04-26T18:18:14.673 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:14 smithi120 ceph-mon[32244]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:14.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:14 smithi037 ceph-mon[29171]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:14.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:14 smithi170 ceph-mon[32697]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:16.669 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:16 smithi120 ceph-mon[32244]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:16.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:16 smithi037 ceph-mon[29171]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:16.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:16 smithi170 ceph-mon[32697]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:18 smithi037 ceph-mon[29171]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:18 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:18 smithi120 ceph-mon[32244]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:18 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:18.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:18 smithi170 ceph-mon[32697]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:18.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:18.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:18 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:20.666 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:20 smithi120 ceph-mon[32244]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:20.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:20 smithi037 ceph-mon[29171]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:20.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.752+0000 7f8775ffb700 1 -- 172.21.15.120:0/2336772853 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8778063860 con 0x7f87680a3c40 2024-04-26T18:18:20.754 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 7 on host 'smithi120' 2024-04-26T18:18:20.755 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 msgr2=0x7f87680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 0x7f87680a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f876c0038c0 tx=0x7f876c00b540 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 msgr2=0x7f87781a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8770007ee0 tx=0x7f877000d3b0 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 shutdown_connections 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87680a3c40 0x7f87680a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8778103f10 0x7f87781a0970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8778102370 0x7f877806c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 --2- 172.21.15.120:0/2336772853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8778103570 0x7f877819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:20.756 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 >> 172.21.15.120:0/2336772853 conn(0x7f87780fd8f0 msgr2=0x7f8778106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:20.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 shutdown_connections 2024-04-26T18:18:20.757 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:18:20.755+0000 7f877efc6700 1 -- 172.21.15.120:0/2336772853 wait complete. 2024-04-26T18:18:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:20 smithi170 ceph-mon[32697]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:21.428 DEBUG:teuthology.orchestra.run.smithi120:osd.7> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T18:18:21.430 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2024-04-26T18:18:21.430 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:18:21.678 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:21.801 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.802 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:21 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:21.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:21 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:18:22.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:22.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:21 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:22.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:22 smithi170 ceph-mon[32697]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:18:22.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:22 smithi170 ceph-mon[32697]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:22 smithi170 ceph-mon[32697]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:18:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:22 smithi170 ceph-mon[32697]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:22.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:22 smithi037 ceph-mon[29171]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:18:22.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:22 smithi037 ceph-mon[29171]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:22.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:22 smithi037 ceph-mon[29171]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:18:22.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:22.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:22 smithi037 ceph-mon[29171]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:23.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-mon[32244]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:18:23.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-mon[32244]: from='osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:23.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-mon[32244]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:18:23.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:23.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-mon[32244]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:18:23.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:18:22.714+0000 7f34b50d6700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T18:18:23.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:18:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:18:22.723+0000 7f34b0ec9700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:18:23.685 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:18:23.685 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:18:23.685 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-26T18:18:23.686 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-26T18:18:23.686 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00868999 s, 1.2 GB/s 2024-04-26T18:18:23.686 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-26T18:18:23.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:23 smithi037 ceph-mon[29171]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:23.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:23 smithi037 ceph-mon[29171]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:18:23.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:23 smithi037 ceph-mon[29171]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:18:23.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:23.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:23 smithi120 ceph-mon[32244]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:24.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:23 smithi120 ceph-mon[32244]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:18:24.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:23 smithi120 ceph-mon[32244]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:18:24.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:23 smithi170 ceph-mon[32697]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:18:24.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:23 smithi170 ceph-mon[32697]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:18:24.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:23 smithi170 ceph-mon[32697]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:18:24.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.035 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-04-26T18:18:24.572 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] boot 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.924 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:24 smithi170 ceph-mon[32697]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:18:24.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:18:24.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:18:24.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] boot 2024-04-26T18:18:24.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T18:18:24.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:24.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:24 smithi037 ceph-mon[29171]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:18:25.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:18:25.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:18:25.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: osd.7 [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] boot 2024-04-26T18:18:25.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T18:18:25.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:18:25.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:24 smithi120 ceph-mon[32244]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi120 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:25.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 5696M 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:25.956 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:25 smithi170 ceph-mon[32697]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi120 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 5696M 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:26.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:26.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:26.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:25 smithi120 ceph-mon[32244]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:18:26.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.149+0000 7f0bf97a3700 1 -- 172.21.15.170:0/1934678553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 msgr2=0x7f0bf410a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:26.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.149+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/1934678553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf410a7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00b0a0 tx=0x7f0bec00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.149+0000 7f0bf97a3700 1 -- 172.21.15.170:0/1934678553 shutdown_connections 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.149+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/1934678553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf410a7d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/1934678553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 0x7f0bf41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/1934678553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 0x7f0bf41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 -- 172.21.15.170:0/1934678553 >> 172.21.15.170:0/1934678553 conn(0x7f0bf40fd920 msgr2=0x7f0bf40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 -- 172.21.15.170:0/1934678553 shutdown_connections 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 -- 172.21.15.170:0/1934678553 wait complete. 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 Processor -- start 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 -- start start 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 0x7f0bf406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.150+0000 7f0bf97a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 0x7f0bf419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410de20 con 0x7f0bf4103580 2024-04-26T18:18:26.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410dca0 con 0x7f0bf4102380 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410dfa0 con 0x7f0bf4103f20 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35534/0 (socket says 172.21.15.170:35534) 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 -- 172.21.15.170:0/3740948000 learned_addr learned my addr 172.21.15.170:0/3740948000 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf27fc700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 0x7f0bf419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf2ffd700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 0x7f0bf406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 -- 172.21.15.170:0/3740948000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 msgr2=0x7f0bf406bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 0x7f0bf406bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 -- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 msgr2=0x7f0bf419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 0x7f0bf419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 -- 172.21.15.170:0/3740948000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf41a1230 con 0x7f0bf4103f20 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf37fe700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009be0 tx=0x7f0bec004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:26.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec021070 con 0x7f0bf4103f20 2024-04-26T18:18:26.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf41a14c0 con 0x7f0bf4103f20 2024-04-26T18:18:26.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec003c00 con 0x7f0bf4103f20 2024-04-26T18:18:26.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0047b0 con 0x7f0bf4103f20 2024-04-26T18:18:26.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.151+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a1910 con 0x7f0bf4103f20 2024-04-26T18:18:26.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.152+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf402e490 con 0x7f0bf4103f20 2024-04-26T18:18:26.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.154+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0bec003db0 con 0x7f0bf4103f20 2024-04-26T18:18:26.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.155+0000 7f0bdbfff700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 0x7f0bdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:26.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.155+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f0bec00f020 con 0x7f0bf4103f20 2024-04-26T18:18:26.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.155+0000 7f0bf2ffd700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 0x7f0bdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:26.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.155+0000 7f0bf2ffd700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 0x7f0bdc0a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0bf41033e0 tx=0x7f0be4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:26.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.158+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bec02a020 con 0x7f0bf4103f20 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi120 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 5696M 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:25 smithi037 ceph-mon[29171]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:18:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:26.366+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0bf4063860 con 0x7f0bdc0a3c40 2024-04-26T18:18:27.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:26 smithi120 ceph-mon[32244]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:26 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:26 smithi120 ceph-mon[32244]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:18:27.005 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:26 smithi170 ceph-mon[32697]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:27.005 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:27.006 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:27.006 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:26 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.006 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:26 smithi170 ceph-mon[32697]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:18:27.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:26 smithi037 ceph-mon[29171]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:26 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:26 smithi037 ceph-mon[29171]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:18:27.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:27 smithi170 ceph-mon[32697]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:27.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:27 smithi170 ceph-mon[32697]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:18:28.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:27 smithi120 ceph-mon[32244]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:28.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:27 smithi120 ceph-mon[32244]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:18:28.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:27 smithi037 ceph-mon[29171]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:28.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:27 smithi037 ceph-mon[29171]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:18:30.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/358244273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]': finished 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:29 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/658540518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:30.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:30.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/358244273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]': finished 2024-04-26T18:18:30.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T18:18:30.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:30.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:29 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/658540518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/358244273' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]: dispatch 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8988738f-786a-4a7e-90da-7bab9c98319f"}]': finished 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:30.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:29 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/658540518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:32.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:31 smithi120 ceph-mon[32244]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:32.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:31 smithi170 ceph-mon[32697]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:32.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:31 smithi037 ceph-mon[29171]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:34.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:33 smithi120 ceph-mon[32244]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:33 smithi170 ceph-mon[32697]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:34.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:33 smithi037 ceph-mon[29171]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:35 smithi120 ceph-mon[32244]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:36.031 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:35 smithi170 ceph-mon[32697]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:36.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:35 smithi037 ceph-mon[29171]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:36.960 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:18:36.960 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:36.960 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:36 smithi170 ceph-mon[32697]: Deploying daemon osd.8 on smithi170 2024-04-26T18:18:37.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:18:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:36 smithi120 ceph-mon[32244]: Deploying daemon osd.8 on smithi170 2024-04-26T18:18:37.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:18:37.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:37.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:36 smithi037 ceph-mon[29171]: Deploying daemon osd.8 on smithi170 2024-04-26T18:18:37.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:37 smithi170 ceph-mon[32697]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:38.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:37 smithi037 ceph-mon[29171]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:38.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:37 smithi120 ceph-mon[32244]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:40.054 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:39 smithi120 ceph-mon[32244]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:40.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:39 smithi170 ceph-mon[32697]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:40.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:39 smithi037 ceph-mon[29171]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:42.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:41 smithi170 ceph-mon[32697]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:42.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:41 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:42.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:41 smithi037 ceph-mon[29171]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:41 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:41 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:41 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:42.251 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:41 smithi120 ceph-mon[32244]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:42.251 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:41 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.251 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:41 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:42.251 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:41 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:44.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:43 smithi170 ceph-mon[32697]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:44.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:43 smithi037 ceph-mon[29171]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:44.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:43 smithi120 ceph-mon[32244]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.284+0000 7f0bdbfff700 1 -- 172.21.15.170:0/3740948000 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0bf4063860 con 0x7f0bdc0a3c40 2024-04-26T18:18:45.284 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2024-04-26T18:18:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 msgr2=0x7f0bdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 0x7f0bdc0a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0bf41033e0 tx=0x7f0be4009260 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 msgr2=0x7f0bf41a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009be0 tx=0x7f0bec004ce0 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 shutdown_connections 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0bdc0a3c40 0x7f0bdc0a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bf4103f20 0x7f0bf41a0960 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0bf4102380 0x7f0bf406bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 --2- 172.21.15.170:0/3740948000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf4103580 0x7f0bf419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 >> 172.21.15.170:0/3740948000 conn(0x7f0bf40fd920 msgr2=0x7f0bf41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.286+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 shutdown_connections 2024-04-26T18:18:45.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:18:45.287+0000 7f0bf97a3700 1 -- 172.21.15.170:0/3740948000 wait complete. 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:45 smithi037 ceph-mon[29171]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:45 smithi120 ceph-mon[32244]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:45 smithi170 ceph-mon[32697]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:18:45.892 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T18:18:45.895 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-26T18:18:45.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:18:46.151 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:46.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:46 smithi037 ceph-mon[29171]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:46.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:18:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:18:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:46.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:46 smithi120 ceph-mon[32244]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:46.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:46.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:18:46.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: from='osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:46.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:18:46.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:46.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:46 smithi170 ceph-mon[32697]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:18:47.257 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:18:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:18:47.244+0000 7eff1bd49700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T18:18:47.257 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:18:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:18:47.256+0000 7eff17b3c700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:18:48.286 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:18:48.286 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:18:48.287 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T18:18:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T18:18:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127349 s, 823 MB/s 2024-04-26T18:18:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T18:18:48.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:48 smithi170 ceph-mon[32697]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:48.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:48 smithi170 ceph-mon[32697]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:18:48.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:48 smithi170 ceph-mon[32697]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:18:48.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:48 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:48 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:48 smithi120 ceph-mon[32244]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:48 smithi120 ceph-mon[32244]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:18:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:48 smithi120 ceph-mon[32244]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:18:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:48 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:48 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:48 smithi037 ceph-mon[29171]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:18:48.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:48 smithi037 ceph-mon[29171]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:18:48.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:48 smithi037 ceph-mon[29171]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:18:48.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:48 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:48 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:48.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-26T18:18:49.205 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] boot 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] boot 2024-04-26T18:18:49.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:49.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: osd.8 [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] boot 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:49.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:49.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:49.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:49.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:18:50.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:50 smithi037 ceph-mon[29171]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:50.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:50 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi170 2024-04-26T18:18:50.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:50 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-26T18:18:50.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:50 smithi037 ceph-mon[29171]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:18:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:50 smithi170 ceph-mon[32697]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:50 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi170 2024-04-26T18:18:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:50 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-26T18:18:50.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:50 smithi170 ceph-mon[32697]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:18:50.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:50 smithi120 ceph-mon[32244]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:50.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:50 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi170 2024-04-26T18:18:50.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:50 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-26T18:18:50.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:50 smithi120 ceph-mon[32244]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 -- 172.21.15.37:0/1227131198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8104680 msgr2=0x7efdb8104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/1227131198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8104680 0x7efdb8104ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efdac0099c0 tx=0x7efdac009cd0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 -- 172.21.15.37:0/1227131198 shutdown_connections 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/1227131198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8105020 0x7efdb810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/1227131198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 0x7efdb81038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/1227131198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8104680 0x7efdb8104ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 -- 172.21.15.37:0/1227131198 >> 172.21.15.37:0/1227131198 conn(0x7efdb80fea20 msgr2=0x7efdb8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 -- 172.21.15.37:0/1227131198 shutdown_connections 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.858+0000 7efdbc8b1700 1 -- 172.21.15.37:0/1227131198 wait complete. 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 Processor -- start 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 -- start start 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 0x7efdb81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8104680 0x7efdb8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810ef20 con 0x7efdb8105020 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810eda0 con 0x7efdb8104680 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdbc8b1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810f0a0 con 0x7efdb8103480 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:50.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37500/0 (socket says 172.21.15.37:37500) 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 -- 172.21.15.37:0/3617036547 learned_addr learned my addr 172.21.15.37:0/3617036547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 0x7efdb81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb5d9b700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8104680 0x7efdb8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 -- 172.21.15.37:0/3617036547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 msgr2=0x7efdb81995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 0x7efdb81995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 -- 172.21.15.37:0/3617036547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8104680 msgr2=0x7efdb8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8104680 0x7efdb8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.859+0000 7efdb6d9d700 1 -- 172.21.15.37:0/3617036547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a1240 con 0x7efdb8105020 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efdb6d9d700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efdb0007f10 tx=0x7efdb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ddd0 con 0x7efdb8105020 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb0004020 con 0x7efdb8105020 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a1530 con 0x7efdb8105020 2024-04-26T18:18:50.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0015660 con 0x7efdb8105020 2024-04-26T18:18:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.860+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a1a70 con 0x7efdb8105020 2024-04-26T18:18:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.861+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb806a110 con 0x7efdb8105020 2024-04-26T18:18:50.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.862+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efdb00044f0 con 0x7efdb8105020 2024-04-26T18:18:50.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.862+0000 7efda77fe700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 0x7efda00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:50.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.863+0000 7efdb659c700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 0x7efda00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:50.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.863+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7efdb00cf440 con 0x7efdb8105020 2024-04-26T18:18:50.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.863+0000 7efdb659c700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 0x7efda00a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efdb81044e0 tx=0x7efda800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:50.868+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdb0010070 con 0x7efdb8105020 2024-04-26T18:18:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:18:51.086+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efdb8063860 con 0x7efda00a3cc0 2024-04-26T18:18:51.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:51 smithi037 ceph-mon[29171]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:18:51.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:51 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:51.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:51 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:51.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:51 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:51 smithi170 ceph-mon[32697]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:18:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:51 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:51 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:51 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:51.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:51 smithi120 ceph-mon[32244]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:18:51.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:51 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:51.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:51 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:51.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:51 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:52 smithi170 ceph-mon[32697]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:52 smithi170 ceph-mon[32697]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:52 smithi170 ceph-mon[32697]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:18:52.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:52 smithi037 ceph-mon[29171]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:52.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:52 smithi037 ceph-mon[29171]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:52.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:52 smithi037 ceph-mon[29171]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:18:52.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:52 smithi120 ceph-mon[32244]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:52.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:52 smithi120 ceph-mon[32244]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:52.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:52 smithi120 ceph-mon[32244]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:18:53.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:53 smithi037 ceph-mon[29171]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:18:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:53 smithi170 ceph-mon[32697]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:18:53.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:53 smithi120 ceph-mon[32244]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:18:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:54 smithi037 ceph-mon[29171]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:54 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]: dispatch 2024-04-26T18:18:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:54 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]': finished 2024-04-26T18:18:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:54 smithi037 ceph-mon[29171]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T18:18:54.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:54 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:18:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:54 smithi170 ceph-mon[32697]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:54 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]: dispatch 2024-04-26T18:18:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:54 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]': finished 2024-04-26T18:18:54.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:54 smithi170 ceph-mon[32697]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T18:18:54.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:54 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:18:54.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:54 smithi120 ceph-mon[32244]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:18:54.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:54 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]: dispatch 2024-04-26T18:18:54.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:54 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2903601609' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c35fc73f-65b6-4db3-8180-ee78649aab29"}]': finished 2024-04-26T18:18:54.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:54 smithi120 ceph-mon[32244]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T18:18:54.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:54 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:18:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:55 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2253745952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:55.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:55 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2253745952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:55.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:55 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2253745952' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:56.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:56 smithi170 ceph-mon[32697]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T18:18:56.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:56 smithi037 ceph-mon[29171]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T18:18:56.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:56 smithi120 ceph-mon[32244]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T18:18:58.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:18:58 smithi037 ceph-mon[29171]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:18:58.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:18:58 smithi170 ceph-mon[32697]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:18:58.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:18:58 smithi120 ceph-mon[32244]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:19:00.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:00 smithi037 ceph-mon[29171]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T18:19:00.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:00 smithi170 ceph-mon[32697]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T18:19:00.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:00 smithi120 ceph-mon[32244]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T18:19:02.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:02 smithi037 ceph-mon[29171]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T18:19:02.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:19:02.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:02 smithi170 ceph-mon[32697]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T18:19:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:19:02.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:02.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:02 smithi120 ceph-mon[32244]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T18:19:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:19:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:03.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:03 smithi037 ceph-mon[29171]: Deploying daemon osd.9 on smithi037 2024-04-26T18:19:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:03 smithi170 ceph-mon[32697]: Deploying daemon osd.9 on smithi037 2024-04-26T18:19:03.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:03 smithi120 ceph-mon[32244]: Deploying daemon osd.9 on smithi037 2024-04-26T18:19:04.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:04 smithi170 ceph-mon[32697]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:19:04.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:04 smithi037 ceph-mon[29171]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:19:04.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:04 smithi120 ceph-mon[32244]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:19:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:06 smithi170 ceph-mon[32697]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:06.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:06 smithi037 ceph-mon[29171]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:06.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:06 smithi120 ceph-mon[32244]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:08.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:08 smithi170 ceph-mon[32697]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:08.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:08 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:08.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:08 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:08.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:08 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:08.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:08 smithi037 ceph-mon[29171]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:08.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:08.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:08.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:08 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:08.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:08 smithi120 ceph-mon[32244]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:08.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:08 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:08.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:08 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:08.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:08 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:10.545 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:10 smithi037 ceph-mon[29171]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:10.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:10 smithi120 ceph-mon[32244]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:10.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:10 smithi170 ceph-mon[32697]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:11.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.299+0000 7efda77fe700 1 -- 172.21.15.37:0/3617036547 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efdb8063860 con 0x7efda00a3cc0 2024-04-26T18:19:11.299 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-04-26T18:19:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 msgr2=0x7efda00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 0x7efda00a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efdb81044e0 tx=0x7efda800b410 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 msgr2=0x7efdb81a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efdb0007f10 tx=0x7efdb000d3b0 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 shutdown_connections 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efda00a3cc0 0x7efda00a6180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdb8105020 0x7efdb81a0ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efdb8103480 0x7efdb81995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 --2- 172.21.15.37:0/3617036547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdb8104680 0x7efdb8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 >> 172.21.15.37:0/3617036547 conn(0x7efdb80fea20 msgr2=0x7efdb8107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 shutdown_connections 2024-04-26T18:19:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:19:11.301+0000 7efdbc8b1700 1 -- 172.21.15.37:0/3617036547 wait complete. 2024-04-26T18:19:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:11.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:11.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:11.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:12.073 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T18:19:12.075 INFO:tasks.cephadm:Deploying osd.10 on smithi120 with /dev/vg_nvme/lv_1... 2024-04-26T18:19:12.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:19:12.321 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:19:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-mon[29171]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-mon[29171]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:19:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-mon[29171]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:19:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-mon[29171]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:19:12.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:12.733 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:19:12.430+0000 7fed45912700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T18:19:12.733 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:19:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:19:12.445+0000 7fed41705700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:19:12.748 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:12 smithi120 ceph-mon[32244]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:12.748 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:12 smithi120 ceph-mon[32244]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:19:12.748 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:12 smithi120 ceph-mon[32244]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:19:12.748 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:12 smithi120 ceph-mon[32244]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:19:12.749 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:12.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:12 smithi170 ceph-mon[32697]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:12.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:12 smithi170 ceph-mon[32697]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:19:12.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:12 smithi170 ceph-mon[32697]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:19:12.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:12 smithi170 ceph-mon[32697]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T18:19:12.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:13 smithi037 ceph-mon[29171]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:19:13.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:13 smithi037 ceph-mon[29171]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:19:13.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:13 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:13 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.645 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:13 smithi120 ceph-mon[32244]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:19:13.645 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:13 smithi120 ceph-mon[32244]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:19:13.646 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:13 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.646 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:13 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:13 smithi170 ceph-mon[32697]: from='osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T18:19:13.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:13 smithi170 ceph-mon[32697]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:19:13.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:13 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:13.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:13 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.242 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:19:14.243 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:19:14.243 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-04-26T18:19:14.243 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-04-26T18:19:14.243 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130648 s, 803 MB/s 2024-04-26T18:19:14.243 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] boot 2024-04-26T18:19:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T18:19:14.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] boot 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T18:19:14.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:19:14.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:19:14.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:19:14.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: osd.9 [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] boot 2024-04-26T18:19:14.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T18:19:14.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:19:14.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-04-26T18:19:15.093 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:15.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:15.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:19:15.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:15.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:16.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:16 smithi120 ceph-mon[32244]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:16.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:16 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi037 2024-04-26T18:19:16.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:16 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T18:19:16.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:16 smithi037 ceph-mon[29171]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:16.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:16 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi037 2024-04-26T18:19:16.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:16 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T18:19:16.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.789+0000 7f1730bdb700 1 -- 172.21.15.120:0/1667594262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 msgr2=0x7f172c1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:16.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.789+0000 7f1730bdb700 1 --2- 172.21.15.120:0/1667594262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c1027a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f171c0099c0 tx=0x7f171c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.790 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 -- 172.21.15.120:0/1667594262 shutdown_connections 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 --2- 172.21.15.120:0/1667594262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103f20 0x7f172c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 --2- 172.21.15.120:0/1667594262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c1027a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 --2- 172.21.15.120:0/1667594262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103580 0x7f172c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 -- 172.21.15.120:0/1667594262 >> 172.21.15.120:0/1667594262 conn(0x7f172c0fd940 msgr2=0x7f172c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 -- 172.21.15.120:0/1667594262 shutdown_connections 2024-04-26T18:19:16.791 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 -- 172.21.15.120:0/1667594262 wait complete. 2024-04-26T18:19:16.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 Processor -- start 2024-04-26T18:19:16.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 -- start start 2024-04-26T18:19:16.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:16.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.790+0000 7f1730bdb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 0x7f172c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:16.795 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 0x7f172c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:16.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10de70 con 0x7f172c103580 2024-04-26T18:19:16.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10dcf0 con 0x7f172c102380 2024-04-26T18:19:16.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f172c10dff0 con 0x7f172c103f20 2024-04-26T18:19:16.796 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:52150/0 (socket says 172.21.15.120:52150) 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 -- 172.21.15.120:0/2400646236 learned_addr learned my addr 172.21.15.120:0/2400646236 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1729d9b700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 0x7f172c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172ad9d700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 0x7f172c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 -- 172.21.15.120:0/2400646236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 msgr2=0x7f172c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:16.797 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 0x7f172c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 -- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 msgr2=0x7f172c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 0x7f172c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 -- 172.21.15.120:0/2400646236 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c1a1300 con 0x7f172c102380 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172ad9d700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 0x7f172c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1729d9b700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 0x7f172c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:19:16.798 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f172a59c700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f171c009f50 tx=0x7f171c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c027070 con 0x7f172c102380 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171c022470 con 0x7f172c102380 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1a1590 con 0x7f172c102380 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c00f6c0 con 0x7f172c102380 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.791+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1a1a70 con 0x7f172c102380 2024-04-26T18:19:16.799 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.792+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c06a110 con 0x7f172c102380 2024-04-26T18:19:16.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.793+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f171c02a070 con 0x7f172c102380 2024-04-26T18:19:16.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.794+0000 7f171b7fe700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 0x7f17140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:16.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.794+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f171c0d0f10 con 0x7f172c102380 2024-04-26T18:19:16.800 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.794+0000 7f1729d9b700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 0x7f17140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:16.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.794+0000 7f1729d9b700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 0x7f17140a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f172c06cc70 tx=0x7f1720008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:16.801 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:16.797+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f171c031080 con 0x7f172c102380 2024-04-26T18:19:16.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:16 smithi170 ceph-mon[32697]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:16.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:16 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi037 2024-04-26T18:19:16.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:16 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T18:19:17.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:17.004+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f172c063920 con 0x7f17140a3c40 2024-04-26T18:19:17.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:17.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:17.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:17.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:17.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:17.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:17.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:18.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:18 smithi037 ceph-mon[29171]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:18.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:18 smithi037 ceph-mon[29171]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:18.749 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:18 smithi120 ceph-mon[32244]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:18.749 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:18 smithi120 ceph-mon[32244]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:18.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:18 smithi170 ceph-mon[32697]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:18.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:18 smithi170 ceph-mon[32697]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/1571924671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]': finished 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:20 smithi037 ceph-mon[29171]: from='client.? 172.21.15.120:0/3405538672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/1571924671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]': finished 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:20.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:20 smithi120 ceph-mon[32244]: from='client.? 172.21.15.120:0/3405538672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/1571924671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]: dispatch 2024-04-26T18:19:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "416522bb-db89-4bfd-83ca-082a3abd1dcf"}]': finished 2024-04-26T18:19:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T18:19:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:20 smithi170 ceph-mon[32697]: from='client.? 172.21.15.120:0/3405538672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:22 smithi037 ceph-mon[29171]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:22.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:22 smithi120 ceph-mon[32244]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:22.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:22 smithi170 ceph-mon[32697]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:24 smithi037 ceph-mon[29171]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:24.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:24 smithi120 ceph-mon[32244]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:24.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:24 smithi170 ceph-mon[32697]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:26 smithi120 ceph-mon[32244]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:26 smithi170 ceph-mon[32697]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:26.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:26 smithi037 ceph-mon[29171]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:27.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:27 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:19:27.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:27 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:27.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:27 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:19:27.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:27 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:27 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:27 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:28 smithi120 ceph-mon[32244]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:28.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:28 smithi120 ceph-mon[32244]: Deploying daemon osd.10 on smithi120 2024-04-26T18:19:28.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:28 smithi170 ceph-mon[32697]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:28.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:28 smithi170 ceph-mon[32697]: Deploying daemon osd.10 on smithi120 2024-04-26T18:19:28.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:28 smithi037 ceph-mon[29171]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:28.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:28 smithi037 ceph-mon[29171]: Deploying daemon osd.10 on smithi120 2024-04-26T18:19:30.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:30 smithi120 ceph-mon[32244]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:30.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:30 smithi170 ceph-mon[32697]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:30.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:30 smithi037 ceph-mon[29171]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:32.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:32 smithi120 ceph-mon[32244]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:32.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:32 smithi170 ceph-mon[32697]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:32.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:32 smithi037 ceph-mon[29171]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:33.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:33.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:33.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:33 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:33.941 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:33 smithi120 ceph-mon[32244]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:33.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:33.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:33.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:33 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:33.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:33 smithi037 ceph-mon[29171]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:33 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:33 smithi170 ceph-mon[32697]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:36.371 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.371+0000 7f171b7fe700 1 -- 172.21.15.120:0/2400646236 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f172c063920 con 0x7f17140a3c40 2024-04-26T18:19:36.372 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 10 on host 'smithi120' 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 msgr2=0x7f17140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 0x7f17140a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f172c06cc70 tx=0x7f1720008040 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 msgr2=0x7f172c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f171c009f50 tx=0x7f171c009510 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 shutdown_connections 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17140a3c40 0x7f17140a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f172c103f20 0x7f172c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f172c102380 0x7f172c06c710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 --2- 172.21.15.120:0/2400646236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f172c103580 0x7f172c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.373+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 >> 172.21.15.120:0/2400646236 conn(0x7f172c0fd940 msgr2=0x7f172c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:36.374 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.374+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 shutdown_connections 2024-04-26T18:19:36.375 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T18:19:36.374+0000 7f1730bdb700 1 -- 172.21.15.120:0/2400646236 wait complete. 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:36 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.745 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:36 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:36.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:36 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:37.080 DEBUG:teuthology.orchestra.run.smithi120:osd.10> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T18:19:37.082 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2024-04-26T18:19:37.082 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:19:37.336 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:19:37.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:37 smithi120 ceph-mon[32244]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:37.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:37 smithi120 ceph-mon[32244]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:37.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:37 smithi170 ceph-mon[32697]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:37.683 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:37 smithi170 ceph-mon[32697]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:37.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:37 smithi037 ceph-mon[29171]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:37.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:37 smithi037 ceph-mon[29171]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:19:38.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:38.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:19:38.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:19:38.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:38.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:38.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-mon[32244]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:38.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:19:38.449+0000 7fe592ce6700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T18:19:38.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:19:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:19:38.465+0000 7fe58ead9700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:38.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:38 smithi170 ceph-mon[32697]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:38.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:19:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:19:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: from='osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:38.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:38 smithi037 ceph-mon[29171]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00830729 s, 1.3 GB/s 2024-04-26T18:19:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-26T18:19:39.674 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:39 smithi120 ceph-mon[32244]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:19:39.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:39 smithi120 ceph-mon[32244]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:19:39.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.675 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:39 smithi037 ceph-mon[29171]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:19:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:39 smithi037 ceph-mon[29171]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:19:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:39 smithi170 ceph-mon[32697]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-04-26T18:19:39.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:39 smithi170 ceph-mon[32697]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:19:39.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.821 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:39.848 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-04-26T18:19:40.086 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:19:40.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:19:40.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:19:40.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:40.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:40.523 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] boot 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:40.524 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:40.525 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] boot 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:19:40.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:40.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:19:42.184 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:19:42.184 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: osd.10 [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] boot 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.185 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:42.186 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:41 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi120 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:41 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 4272M 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:41 smithi037 ceph-mon[29171]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:41 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi120 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:41 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 4272M 2024-04-26T18:19:42.187 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:41 smithi120 ceph-mon[32244]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 -- 172.21.15.170:0/4259421222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e81068e0 msgr2=0x7f78e8108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 --2- 172.21.15.170:0/4259421222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e81068e0 0x7f78e8108cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009990 tx=0x7f78d0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 -- 172.21.15.170:0/4259421222 shutdown_connections 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 --2- 172.21.15.170:0/4259421222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 0x7f78e810b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 --2- 172.21.15.170:0/4259421222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e8103fb0 0x7f78e81063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 --2- 172.21.15.170:0/4259421222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e81068e0 0x7f78e8108cd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 -- 172.21.15.170:0/4259421222 >> 172.21.15.170:0/4259421222 conn(0x7f78e80fd920 msgr2=0x7f78e80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 -- 172.21.15.170:0/4259421222 shutdown_connections 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 -- 172.21.15.170:0/4259421222 wait complete. 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.674+0000 7f78ec858700 1 Processor -- start 2024-04-26T18:19:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 -- start start 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e81068e0 0x7f78e8101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 0x7f78e81024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810ef80 con 0x7f78e81068e0 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810ee00 con 0x7f78e8109210 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ec858700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810f100 con 0x7f78e8103fb0 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e6d9d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 0x7f78e81024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36232/0 (socket says 172.21.15.170:36232) 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 -- 172.21.15.170:0/3798532586 learned_addr learned my addr 172.21.15.170:0/3798532586 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78ddd9b700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e81068e0 0x7f78e8101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 -- 172.21.15.170:0/3798532586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 msgr2=0x7f78e81024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 0x7f78e81024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 -- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e81068e0 msgr2=0x7f78e8101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e81068e0 0x7f78e8101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.675+0000 7f78e659c700 1 -- 172.21.15.170:0/3798532586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e8102d50 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78e659c700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b530 tx=0x7f78d800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8015020 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d8011ce0 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e81a3810 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010dd0 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.676+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a3da0 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.677+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e806a110 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.678+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78d8018020 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.678+0000 7f78dffff700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 0x7f78d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.679+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f78d80cfa40 con 0x7f78e8103fb0 2024-04-26T18:19:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.679+0000 7f78ddd9b700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 0x7f78d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.679+0000 7f78ddd9b700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 0x7f78d40a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b5c0 tx=0x7f78d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.682+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78d801f020 con 0x7f78e8103fb0 2024-04-26T18:19:42.193 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:41 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi120 2024-04-26T18:19:42.193 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:41 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 4272M 2024-04-26T18:19:42.193 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:41 smithi170 ceph-mon[32697]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:19:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:19:41.887+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f78e8063860 con 0x7f78d40a3c40 2024-04-26T18:19:42.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:42 smithi170 ceph-mon[32697]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:42.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:42.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:42.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:42 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:42 smithi170 ceph-mon[32697]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:19:42.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:42 smithi037 ceph-mon[29171]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:42.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:42.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:42.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:42 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:42 smithi037 ceph-mon[29171]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:19:42.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:42 smithi120 ceph-mon[32244]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:42.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:42.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:42.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:42 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:42 smithi120 ceph-mon[32244]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:19:43.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:43 smithi037 ceph-mon[29171]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:43 smithi037 ceph-mon[29171]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:19:43.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:43 smithi120 ceph-mon[32244]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:43 smithi120 ceph-mon[32244]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:19:43.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:43 smithi170 ceph-mon[32697]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:43.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:43 smithi170 ceph-mon[32697]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:19:44.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:44 smithi037 ceph-mon[29171]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:44.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:44 smithi120 ceph-mon[32244]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:44.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:44 smithi170 ceph-mon[32697]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/3448016107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]': finished 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:19:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:45 smithi037 ceph-mon[29171]: from='client.? 172.21.15.170:0/3228585691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/3448016107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]': finished 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:19:45.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:45 smithi120 ceph-mon[32244]: from='client.? 172.21.15.170:0/3228585691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:45.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/3448016107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]: dispatch 2024-04-26T18:19:45.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23"}]': finished 2024-04-26T18:19:45.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T18:19:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:19:45.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:45 smithi170 ceph-mon[32697]: from='client.? 172.21.15.170:0/3228585691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:46.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:46 smithi037 ceph-mon[29171]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-26T18:19:46.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:46 smithi120 ceph-mon[32244]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-26T18:19:46.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:46 smithi170 ceph-mon[32697]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-26T18:19:48.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:48 smithi170 ceph-mon[32697]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:48 smithi037 ceph-mon[29171]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:19:48.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:48 smithi120 ceph-mon[32244]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T18:19:50.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:50 smithi120 ceph-mon[32244]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:19:50.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:50 smithi170 ceph-mon[32697]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:19:50.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:50 smithi037 ceph-mon[29171]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:19:52.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:52 smithi120 ceph-mon[32244]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:19:52.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:19:52.741 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:52 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:52.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:52 smithi170 ceph-mon[32697]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:19:52.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:19:52.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:52 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:52.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:52 smithi037 ceph-mon[29171]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:19:52.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:19:52.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:52 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:53 smithi170 ceph-mon[32697]: Deploying daemon osd.11 on smithi170 2024-04-26T18:19:53.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:53 smithi120 ceph-mon[32244]: Deploying daemon osd.11 on smithi170 2024-04-26T18:19:53.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:53 smithi037 ceph-mon[29171]: Deploying daemon osd.11 on smithi170 2024-04-26T18:19:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:54 smithi170 ceph-mon[32697]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:19:54.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:54 smithi120 ceph-mon[32244]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:19:54.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:54 smithi037 ceph-mon[29171]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:19:56.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:56 smithi170 ceph-mon[32697]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:56.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:56 smithi120 ceph-mon[32244]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:56.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:56 smithi037 ceph-mon[29171]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:58.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:58 smithi120 ceph-mon[32244]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:58.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:19:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:58.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:58 smithi170 ceph-mon[32697]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:58.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:19:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:58 smithi037 ceph-mon[29171]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:19:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:19:58.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:19:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:00 smithi170 ceph-mon[32697]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:00.681 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:00 smithi170 ceph-mon[32697]: overall HEALTH_OK 2024-04-26T18:20:00.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:00 smithi120 ceph-mon[32244]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:00.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:00 smithi120 ceph-mon[32244]: overall HEALTH_OK 2024-04-26T18:20:00.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:00 smithi037 ceph-mon[29171]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:00.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:00 smithi037 ceph-mon[29171]: overall HEALTH_OK 2024-04-26T18:20:01.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.109+0000 7f78dffff700 1 -- 172.21.15.170:0/3798532586 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f78e8063860 con 0x7f78d40a3c40 2024-04-26T18:20:01.110 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2024-04-26T18:20:01.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 msgr2=0x7f78d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:01.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 0x7f78d40a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b5c0 tx=0x7f78d0005fb0 comp rx=0 tx=0).stop 2024-04-26T18:20:01.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 msgr2=0x7f78e8103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b530 tx=0x7f78d800b840 comp rx=0 tx=0).stop 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 shutdown_connections 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f78d40a3c40 0x7f78d40a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78e8109210 0x7f78e81024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78e8103fb0 0x7f78e8103910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:01.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 --2- 172.21.15.170:0/3798532586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78e81068e0 0x7f78e8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:01.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 >> 172.21.15.170:0/3798532586 conn(0x7f78e80fd920 msgr2=0x7f78e8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:01.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 shutdown_connections 2024-04-26T18:20:01.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:01.112+0000 7f78ec858700 1 -- 172.21.15.170:0/3798532586 wait complete. 2024-04-26T18:20:01.871 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T18:20:01.873 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T18:20:01.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T18:20:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:02 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:02 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:02 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-mon[32697]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:20:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-mon[32697]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:20:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:03.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-mon[32697]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-mon[32697]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:03 smithi037 ceph-mon[29171]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:20:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:03 smithi037 ceph-mon[29171]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:20:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:03 smithi037 ceph-mon[29171]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:03 smithi037 ceph-mon[29171]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:03 smithi120 ceph-mon[32244]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:20:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:03 smithi120 ceph-mon[32244]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:20:03.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:03.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:03 smithi120 ceph-mon[32244]: from='osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:03 smithi120 ceph-mon[32244]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-26T18:20:03.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- 172.21.15.37:0/2212270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 msgr2=0x7f3704108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:03.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 --2- 172.21.15.37:0/2212270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704108c30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f36ec009990 tx=0x7f36ec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- 172.21.15.37:0/2212270707 shutdown_connections 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 --2- 172.21.15.37:0/2212270707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f370410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 --2- 172.21.15.37:0/2212270707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 0x7f3704106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 --2- 172.21.15.37:0/2212270707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704108c30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- 172.21.15.37:0/2212270707 >> 172.21.15.37:0/2212270707 conn(0x7f37040fd920 msgr2=0x7f37040ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- 172.21.15.37:0/2212270707 shutdown_connections 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- 172.21.15.37:0/2212270707 wait complete. 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 Processor -- start 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.861+0000 7f370a461700 1 -- start start 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 0x7f3704103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f3704102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f370410ef40 con 0x7f3704106840 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f370410edc0 con 0x7f3704103f10 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f370410f0c0 con 0x7f3704109170 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60108/0 (socket says 172.21.15.37:60108) 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 -- 172.21.15.37:0/3794376927 learned_addr learned my addr 172.21.15.37:0/3794376927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f3703fff700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 0x7f3704103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37089fe700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f3704102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 -- 172.21.15.37:0/3794376927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 msgr2=0x7f3704102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f3704102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 -- 172.21.15.37:0/3794376927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 msgr2=0x7f3704103870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 0x7f3704103870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:03.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 -- 172.21.15.37:0/3794376927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3704102cb0 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37089fe700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f3704102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37037fe700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36ec006010 tx=0x7f36ec00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec027070 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36ec00fc60 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.862+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041a36c0 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.863+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec017650 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.863+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37041a3b10 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.863+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370406a110 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.865+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36ec009d10 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.865+0000 7f37017fa700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 0x7f36f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.865+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f36ec0d1060 con 0x7f3704106840 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.865+0000 7f3703fff700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 0x7f36f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.866+0000 7f3703fff700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 0x7f36f00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36f4005950 tx=0x7f36f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:03.869+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36ec030070 con 0x7f3704106840 2024-04-26T18:20:03.934 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:20:03.815+0000 7fa55e3de700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T18:20:03.934 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:20:03 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:20:03.832+0000 7fa5561c9700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:20:04.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.079+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3704064030 con 0x7f3704106840 2024-04-26T18:20:04.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.079+0000 7f37017fa700 1 -- 172.21.15.37:0/3794376927 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f36ec013070 con 0x7f3704106840 2024-04-26T18:20:04.080 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 msgr2=0x7f36f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 0x7f36f00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36f4005950 tx=0x7f36f40058e0 comp rx=0 tx=0).stop 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 msgr2=0x7f3704101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f36ec006010 tx=0x7f36ec00f510 comp rx=0 tx=0).stop 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 shutdown_connections 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f36f00a3c40 0x7f36f00a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3704109170 0x7f3704102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:04.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3704103f10 0x7f3704103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 --2- 172.21.15.37:0/3794376927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3704106840 0x7f3704101ec0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.082+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 >> 172.21.15.37:0/3794376927 conn(0x7f37040fd920 msgr2=0x7f3704104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.083+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 shutdown_connections 2024-04-26T18:20:04.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:04.083+0000 7f370a461700 1 -- 172.21.15.37:0/3794376927 wait complete. 2024-04-26T18:20:04.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: from='osd.11 ' entity='osd.11' 2024-04-26T18:20:04.188 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:04 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3794376927' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: from='osd.11 ' entity='osd.11' 2024-04-26T18:20:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:04 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3794376927' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: from='osd.11 ' entity='osd.11' 2024-04-26T18:20:04.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:04 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3794376927' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:04.812 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1714155579,"num_in_osds":12,"osd_in_since":1714155584,"num_remapped_pgs":0} 2024-04-26T18:20:05.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: purged_snaps scrub starts 2024-04-26T18:20:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: purged_snaps scrub ok 2024-04-26T18:20:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] boot 2024-04-26T18:20:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T18:20:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:05 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: purged_snaps scrub starts 2024-04-26T18:20:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: purged_snaps scrub ok 2024-04-26T18:20:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] boot 2024-04-26T18:20:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T18:20:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:05 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.549 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: purged_snaps scrub starts 2024-04-26T18:20:05.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: purged_snaps scrub ok 2024-04-26T18:20:05.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: osd.11 [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] boot 2024-04-26T18:20:05.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T18:20:05.550 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:05 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:20:05.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T18:20:06.060 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.394 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.395 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:06 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:06.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:06 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:20:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:06.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:06.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:06 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:07.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:07 smithi037 ceph-mon[29171]: Detected new or changed devices on smithi170 2024-04-26T18:20:07.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:07 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-26T18:20:07.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:07 smithi037 ceph-mon[29171]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:20:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:07 smithi120 ceph-mon[32244]: Detected new or changed devices on smithi170 2024-04-26T18:20:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:07 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-26T18:20:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:07 smithi120 ceph-mon[32244]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:20:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:07 smithi170 ceph-mon[32697]: Detected new or changed devices on smithi170 2024-04-26T18:20:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:07 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-26T18:20:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:07 smithi170 ceph-mon[32697]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:20:07.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 -- 172.21.15.37:0/1057636488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 msgr2=0x7f111410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 --2- 172.21.15.37:0/1057636488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f111410b8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f110c0099c0 tx=0x7f110c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 -- 172.21.15.37:0/1057636488 shutdown_connections 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 --2- 172.21.15.37:0/1057636488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f111410b8f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 --2- 172.21.15.37:0/1057636488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f11141038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 --2- 172.21.15.37:0/1057636488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 0x7f1114104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 -- 172.21.15.37:0/1057636488 >> 172.21.15.37:0/1057636488 conn(0x7f11140fea40 msgr2=0x7f1114100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 -- 172.21.15.37:0/1057636488 shutdown_connections 2024-04-26T18:20:07.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.842+0000 7f1118baa700 1 -- 172.21.15.37:0/1057636488 wait complete. 2024-04-26T18:20:07.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 Processor -- start 2024-04-26T18:20:07.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 -- start start 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 0x7f111419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f111410ef40 con 0x7f1114105040 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f111410edc0 con 0x7f11141046a0 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1118baa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f111410f0c0 con 0x7f11141034a0 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1112d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:07.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f111259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f111259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:54786/0 (socket says 172.21.15.37:54786) 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1112d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58144/0 (socket says 172.21.15.37:58144) 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f111259c700 1 -- 172.21.15.37:0/1127124135 learned_addr learned my addr 172.21.15.37:0/1127124135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.843+0000 7f1111d9b700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 0x7f111419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 -- 172.21.15.37:0/1127124135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 msgr2=0x7f111419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 -- 172.21.15.37:0/1127124135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 msgr2=0x7f111419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 0x7f111419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 -- 172.21.15.37:0/1127124135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141a33f0 con 0x7f1114105040 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f111259c700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1112d9d700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f110c0098e0 tx=0x7f110c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c027070 con 0x7f1114105040 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c00fe00 con 0x7f1114105040 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11141a3680 con 0x7f1114105040 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0173f0 con 0x7f1114105040 2024-04-26T18:20:07.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.844+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11141a3b30 con 0x7f1114105040 2024-04-26T18:20:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.845+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111406a110 con 0x7f1114105040 2024-04-26T18:20:07.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.846+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f110c02a070 con 0x7f1114105040 2024-04-26T18:20:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.847+0000 7f110b7fe700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 0x7f11000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.847+0000 7f111259c700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 0x7f11000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.847+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f110c0d0f70 con 0x7f1114105040 2024-04-26T18:20:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.847+0000 7f111259c700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 0x7f11000a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1104009520 tx=0x7f110400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:07.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:07.851+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f110c013070 con 0x7f1114105040 2024-04-26T18:20:08.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.058+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f11141a3ff0 con 0x7f1114105040 2024-04-26T18:20:08.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.059+0000 7f110b7fe700 1 -- 172.21.15.37:0/1127124135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f11141a3ff0 con 0x7f1114105040 2024-04-26T18:20:08.060 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:08.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 msgr2=0x7f11000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:08.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 0x7f11000a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1104009520 tx=0x7f110400b040 comp rx=0 tx=0).stop 2024-04-26T18:20:08.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 msgr2=0x7f11141a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:08.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f110c0098e0 tx=0x7f110c00fa00 comp rx=0 tx=0).stop 2024-04-26T18:20:08.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 shutdown_connections 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f11000a3cc0 0x7f11000a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1114105040 0x7f11141a2d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11141034a0 0x7f111419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 --2- 172.21.15.37:0/1127124135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11141046a0 0x7f111419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 >> 172.21.15.37:0/1127124135 conn(0x7f11140fea40 msgr2=0x7f1114075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 shutdown_connections 2024-04-26T18:20:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:08.062+0000 7f1118baa700 1 -- 172.21.15.37:0/1127124135 wait complete. 2024-04-26T18:20:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:08 smithi037 ceph-mon[29171]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:08.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:08 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1127124135' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:08.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:08 smithi120 ceph-mon[32244]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:08 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1127124135' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:08.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:08 smithi170 ceph-mon[32697]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:08.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:08 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1127124135' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:20:08.818 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714155604,"num_in_osds":12,"osd_in_since":1714155584,"num_remapped_pgs":0} 2024-04-26T18:20:08.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:20:09.069 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:10.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:10 smithi037 ceph-mon[29171]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:10.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:10 smithi120 ceph-mon[32244]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:10.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:10 smithi170 ceph-mon[32697]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:10.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 -- 172.21.15.37:0/2593926147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 msgr2=0x7f291c108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 --2- 172.21.15.37:0/2593926147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c108770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f290c009990 tx=0x7f290c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 -- 172.21.15.37:0/2593926147 shutdown_connections 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 --2- 172.21.15.37:0/2593926147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 0x7f291c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 --2- 172.21.15.37:0/2593926147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c108770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 --2- 172.21.15.37:0/2593926147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 -- 172.21.15.37:0/2593926147 >> 172.21.15.37:0/2593926147 conn(0x7f291c0792d0 msgr2=0x7f291c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 -- 172.21.15.37:0/2593926147 shutdown_connections 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.822+0000 7f2921dfa700 1 -- 172.21.15.37:0/2593926147 wait complete. 2024-04-26T18:20:10.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 Processor -- start 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 -- start start 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 0x7f291c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1124b0 con 0x7f291c108390 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f291c112330 con 0x7f291c102320 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.823+0000 7f2921dfa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f291c112630 con 0x7f291c102ce0 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:40170/0 (socket says 172.21.15.37:40170) 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 -- 172.21.15.37:0/3281232071 learned_addr learned my addr 172.21.15.37:0/3281232071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:10.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291affd700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 0x7f291c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 -- 172.21.15.37:0/3281232071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 msgr2=0x7f291c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 0x7f291c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 -- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 msgr2=0x7f291c1a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c1a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 -- 172.21.15.37:0/3281232071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1a5eb0 con 0x7f291c102320 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291bfff700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f291b7fe700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:10.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c027070 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c003f20 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1a6080 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c022ca0 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.824+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a64d0 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.825+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c06a110 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.826+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f290c02a070 con 0x7f291c102320 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.827+0000 7f2918ff9700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 0x7f29040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.827+0000 7f291affd700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 0x7f29040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:10.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.827+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f290c0d0dc0 con 0x7f291c102320 2024-04-26T18:20:10.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.828+0000 7f291affd700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 0x7f29040a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f291c0799b0 tx=0x7f29100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:10.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:10.832+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f290c05eff0 con 0x7f291c102320 2024-04-26T18:20:11.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.047+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f291c02e650 con 0x7f291c102320 2024-04-26T18:20:11.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.048+0000 7f2918ff9700 1 -- 172.21.15.37:0/3281232071 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18416 (secure 0 0 0) 0x7f290c013070 con 0x7f291c102320 2024-04-26T18:20:11.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:11.049 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:20:06.117644+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 msgr2=0x7f29040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 0x7f29040a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f291c0799b0 tx=0x7f29100073d0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 msgr2=0x7f291c19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f290c000c00 tx=0x7f290c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 shutdown_connections 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f29040a3c40 0x7f29040a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f291c108390 0x7f291c1a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f291c102320 0x7f291c19a8f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 --2- 172.21.15.37:0/3281232071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f291c102ce0 0x7f291c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:11.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.050+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 >> 172.21.15.37:0/3281232071 conn(0x7f291c0792d0 msgr2=0x7f291c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:11.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.051+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 shutdown_connections 2024-04-26T18:20:11.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:11.051+0000 7f2921dfa700 1 -- 172.21.15.37:0/3281232071 wait complete. 2024-04-26T18:20:11.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:11 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3281232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:20:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:11 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3281232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:20:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:11 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3281232071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:20:11.651 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:16:20.665733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T18:20:11.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T18:20:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:12.325 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:12 smithi037 ceph-mon[29171]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:12 smithi120 ceph-mon[32244]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:12 smithi170 ceph-mon[32697]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:13.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 -- 172.21.15.37:0/2488806592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51401034a0 msgr2=0x7f5140103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 --2- 172.21.15.37:0/2488806592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51401034a0 0x7f5140103920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 -- 172.21.15.37:0/2488806592 shutdown_connections 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 --2- 172.21.15.37:0/2488806592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5140103e60 0x7f51401102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 --2- 172.21.15.37:0/2488806592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51401094e0 0x7f51401098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 --2- 172.21.15.37:0/2488806592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51401034a0 0x7f5140103920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 -- 172.21.15.37:0/2488806592 >> 172.21.15.37:0/2488806592 conn(0x7f514007a590 msgr2=0x7f514007a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 -- 172.21.15.37:0/2488806592 shutdown_connections 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.381+0000 7f5146d8d700 1 -- 172.21.15.37:0/2488806592 wait complete. 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 Processor -- start 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 -- start start 2024-04-26T18:20:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51401034a0 0x7f514019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51401094e0 0x7f5140199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51401135e0 con 0x7f5140103e60 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5140113460 con 0x7f51401034a0 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5146d8d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5140113760 con 0x7f51401094e0 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58182/0 (socket says 172.21.15.37:58182) 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 -- 172.21.15.37:0/3210923513 learned_addr learned my addr 172.21.15.37:0/3210923513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5144b29700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51401034a0 0x7f514019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 -- 172.21.15.37:0/3210923513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51401094e0 msgr2=0x7f5140199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51401094e0 0x7f5140199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 -- 172.21.15.37:0/3210923513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51401034a0 msgr2=0x7f514019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51401034a0 0x7f514019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.382+0000 7f5137fff700 1 -- 172.21.15.37:0/3210923513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514019a0a0 con 0x7f5140103e60 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5137fff700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5130000c00 tx=0x7f51300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130027070 con 0x7f5140103e60 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51300040e0 con 0x7f5140103e60 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514019a330 con 0x7f5140103e60 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513000f460 con 0x7f5140103e60 2024-04-26T18:20:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.383+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401a6650 con 0x7f5140103e60 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.384+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514006a110 con 0x7f5140103e60 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.385+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f513002a070 con 0x7f5140103e60 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.386+0000 7f5135ffb700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 0x7f51280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.386+0000 7f5144b29700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 0x7f51280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.386+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f51300d1380 con 0x7f5140103e60 2024-04-26T18:20:13.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.387+0000 7f5144b29700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 0x7f51280a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f513c005950 tx=0x7f513c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:13.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.390+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5130031070 con 0x7f5140103e60 2024-04-26T18:20:13.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.595+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f514002e490 con 0x7f5140103e60 2024-04-26T18:20:13.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.595+0000 7f5135ffb700 1 -- 172.21.15.37:0/3210923513 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f5130013070 con 0x7f5140103e60 2024-04-26T18:20:13.595 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T18:20:13.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 msgr2=0x7f51280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 0x7f51280a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f513c005950 tx=0x7f513c00a300 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 msgr2=0x7f514019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5130000c00 tx=0x7f51300047b0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 shutdown_connections 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51280a3cc0 0x7f51280a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51401094e0 0x7f5140199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51401034a0 0x7f514019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 --2- 172.21.15.37:0/3210923513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5140103e60 0x7f514019fcc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 >> 172.21.15.37:0/3210923513 conn(0x7f514007a590 msgr2=0x7f5140102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 shutdown_connections 2024-04-26T18:20:13.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:13.597+0000 7f5146d8d700 1 -- 172.21.15.37:0/3210923513 wait complete. 2024-04-26T18:20:14.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:14 smithi037 ceph-mon[29171]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:14.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:14 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3210923513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.b on smithi120 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.e on smithi120 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.h on smithi120 2024-04-26T18:20:14.282 INFO:tasks.cephadm:Adding mds.k on smithi120 2024-04-26T18:20:14.283 INFO:tasks.cephadm:Adding mds.c on smithi170 2024-04-26T18:20:14.283 INFO:tasks.cephadm:Adding mds.f on smithi170 2024-04-26T18:20:14.283 INFO:tasks.cephadm:Adding mds.i on smithi170 2024-04-26T18:20:14.283 INFO:tasks.cephadm:Adding mds.l on smithi170 2024-04-26T18:20:14.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2024-04-26T18:20:14.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:14 smithi120 ceph-mon[32244]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:14 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3210923513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:20:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:20:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:14 smithi170 ceph-mon[32697]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:14 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3210923513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:20:16.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 -- 172.21.15.170:0/202367429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18103480 msgr2=0x7fbd181038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/202367429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18103480 0x7fbd181038a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009990 tx=0x7fbd08009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 -- 172.21.15.170:0/202367429 shutdown_connections 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/202367429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18105020 0x7fbd1810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/202367429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18103480 0x7fbd181038a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/202367429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18104680 0x7fbd18104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 -- 172.21.15.170:0/202367429 >> 172.21.15.170:0/202367429 conn(0x7fbd180fea20 msgr2=0x7fbd18100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 -- 172.21.15.170:0/202367429 shutdown_connections 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.048+0000 7fbd1d96a700 1 -- 172.21.15.170:0/202367429 wait complete. 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.049+0000 7fbd1d96a700 1 Processor -- start 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.049+0000 7fbd1d96a700 1 -- start start 2024-04-26T18:20:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.049+0000 7fbd1d96a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 0x7fbd181996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.049+0000 7fbd1d96a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18104680 0x7fbd18199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810ef80 con 0x7fbd18103480 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810ee00 con 0x7fbd18104680 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810f100 con 0x7fbd18105020 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43308/0 (socket says 172.21.15.170:43308) 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 -- 172.21.15.170:0/3989366587 learned_addr learned my addr 172.21.15.170:0/3989366587 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd16ffd700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 0x7fbd181996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd167fc700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18104680 0x7fbd18199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 -- 172.21.15.170:0/3989366587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18104680 msgr2=0x7fbd18199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18104680 0x7fbd18199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 -- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 msgr2=0x7fbd181996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 0x7fbd181996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 -- 172.21.15.170:0/3989366587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd181a12c0 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd16ffd700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 0x7fbd181996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd177fe700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000b9a0 tx=0x7fbd1000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10019070 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd10014440 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000ce60 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181a15b0 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.050+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a1af0 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.051+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1806a110 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.052+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd1001c070 con 0x7fbd18105020 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.053+0000 7fbd1c968700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.053+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbd100cf340 con 0x7fbd18105020 2024-04-26T18:20:16.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.053+0000 7fbd16ffd700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:16.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.054+0000 7fbd16ffd700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd080038a0 tx=0x7fbd08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:16.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.057+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd1009c250 con 0x7fbd18105020 2024-04-26T18:20:16.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.276+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) -- 0x7fbd18063a60 con 0x7fbd000a3c40 2024-04-26T18:20:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.284+0000 7fbd1c968700 1 -- 172.21.15.170:0/3989366587 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbd18063a60 con 0x7fbd000a3c40 2024-04-26T18:20:16.285 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 msgr2=0x7fbd000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbd080038a0 tx=0x7fbd08005fd0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 msgr2=0x7fbd181a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000b9a0 tx=0x7fbd1000bd60 comp rx=0 tx=0).stop 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 shutdown_connections 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd18105020 0x7fbd181a0c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd18103480 0x7fbd181996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 --2- 172.21.15.170:0/3989366587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd18104680 0x7fbd18199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 >> 172.21.15.170:0/3989366587 conn(0x7fbd180fea20 msgr2=0x7fbd18107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 shutdown_connections 2024-04-26T18:20:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-26T18:20:16.287+0000 7fbd1d96a700 1 -- 172.21.15.170:0/3989366587 wait complete. 2024-04-26T18:20:16.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:16 smithi037 ceph-mon[29171]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:16.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:16 smithi120 ceph-mon[32244]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:16 smithi170 ceph-mon[32697]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:16.991 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T18:20:16.994 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T18:20:17.037 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T18:20:17.080 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T18:20:17.123 DEBUG:teuthology.orchestra.run.smithi120:mds.b> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T18:20:17.125 DEBUG:teuthology.orchestra.run.smithi120:mds.e> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T18:20:17.169 DEBUG:teuthology.orchestra.run.smithi120:mds.h> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T18:20:17.212 DEBUG:teuthology.orchestra.run.smithi120:mds.k> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T18:20:17.256 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T18:20:17.258 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T18:20:17.302 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T18:20:17.346 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T18:20:17.389 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T18:20:17.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:17.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.412 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:17.412 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:17.412 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:17 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:17 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.538 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:17.538 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-04-26T18:20:17.538 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.538 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:17.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:17 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:17.634 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:18.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:18 smithi170 ceph-mon[32697]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:18.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:18 smithi170 ceph-mon[32697]: Deploying daemon mds.l on smithi170 2024-04-26T18:20:18.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:18 smithi037 ceph-mon[29171]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:18.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:18 smithi037 ceph-mon[29171]: Deploying daemon mds.l on smithi170 2024-04-26T18:20:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:18 smithi120 ceph-mon[32244]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:18.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:18 smithi120 ceph-mon[32244]: Deploying daemon mds.l on smithi170 2024-04-26T18:20:19.075 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:18 smithi170 systemd[1]: Starting Ceph mds.l for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 -- 172.21.15.37:0/141654153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4801073a0 msgr2=0x7fb480107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 --2- 172.21.15.37:0/141654153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4801073a0 0x7fb480107780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0099c0 tx=0x7fb47c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 -- 172.21.15.37:0/141654153 shutdown_connections 2024-04-26T18:20:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 --2- 172.21.15.37:0/141654153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb480103030 0x7fb48010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 --2- 172.21.15.37:0/141654153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4801073a0 0x7fb480107780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 --2- 172.21.15.37:0/141654153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 0x7fb480102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 -- 172.21.15.37:0/141654153 >> 172.21.15.37:0/141654153 conn(0x7fb480078190 msgr2=0x7fb4800785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 -- 172.21.15.37:0/141654153 shutdown_connections 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.385+0000 7fb487bea700 1 -- 172.21.15.37:0/141654153 wait complete. 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 Processor -- start 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 -- start start 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 0x7fb480104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 0x7fb480104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801125f0 con 0x7fb480103030 2024-04-26T18:20:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb480112470 con 0x7fb4801073a0 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb487bea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb480112770 con 0x7fb480102670 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb485185700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb485185700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33508/0 (socket says 172.21.15.37:33508) 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.386+0000 7fb485185700 1 -- 172.21.15.37:0/2972541724 learned_addr learned my addr 172.21.15.37:0/2972541724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485986700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 0x7fb480104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb486187700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 0x7fb480104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 -- 172.21.15.37:0/2972541724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 msgr2=0x7fb480104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 0x7fb480104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 -- 172.21.15.37:0/2972541724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 msgr2=0x7fb480104ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 0x7fb480104ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 -- 172.21.15.37:0/2972541724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4801a80c0 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb486187700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 0x7fb480104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb485185700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb47000d980 tx=0x7fb47000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470017070 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4801a83b0 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470004830 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470005440 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.387+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a88f0 con 0x7fb480103030 2024-04-26T18:20:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.388+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb480103a90 con 0x7fb480103030 2024-04-26T18:20:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.389+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4700055e0 con 0x7fb480103030 2024-04-26T18:20:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.390+0000 7fb476ffd700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 0x7fb46c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.390+0000 7fb485986700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 0x7fb46c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.390+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb4700cfe80 con 0x7fb480103030 2024-04-26T18:20:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.391+0000 7fb485986700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 0x7fb46c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb47c003800 tx=0x7fb47c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.395+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb470020020 con 0x7fb480103030 2024-04-26T18:20:19.455 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 podman[62357]: 2024-04-26 18:20:19.086296045 +0000 UTC m=+0.018127394 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:19.455 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 podman[62357]: 2024-04-26 18:20:19.185356787 +0000 UTC m=+0.117188125 container create ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:19.455 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 podman[62357]: 2024-04-26 18:20:19.429615143 +0000 UTC m=+0.361446478 container init ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:20:19.455 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 podman[62357]: 2024-04-26 18:20:19.432157089 +0000 UTC m=+0.363988433 container start ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:20:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.658+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb480105580 con 0x7fb480103030 2024-04-26T18:20:19.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.659+0000 7fb476ffd700 1 -- 172.21.15.37:0/2972541724 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb47009cd00 con 0x7fb480103030 2024-04-26T18:20:19.659 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:19.659 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T18:20:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 msgr2=0x7fb46c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 0x7fb46c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb47c003800 tx=0x7fb47c01a040 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 msgr2=0x7fb480104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb47000d980 tx=0x7fb47000dd40 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 shutdown_connections 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb46c0a3c40 0x7fb46c0a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4801073a0 0x7fb480104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb480102670 0x7fb480104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 --2- 172.21.15.37:0/2972541724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb480103030 0x7fb480104590 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 >> 172.21.15.37:0/2972541724 conn(0x7fb480078190 msgr2=0x7fb48006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 shutdown_connections 2024-04-26T18:20:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:19.661+0000 7fb487bea700 1 -- 172.21.15.37:0/2972541724 wait complete. 2024-04-26T18:20:19.663 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-26T18:20:19.825 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 ceph-mds[62376]: -- 172.21.15.170:0/2797962387 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605af9b1a40 con 0x5605af985000 2024-04-26T18:20:19.825 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l[62372]: starting mds.l at 2024-04-26T18:20:19.825 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 bash[62357]: ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 2024-04-26T18:20:19.826 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:20:19 smithi170 systemd[1]: Started Ceph mds.l for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:20.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:20.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6832/1924785212,v1:172.21.15.170:6833/1924785212] up:boot 2024-04-26T18:20:20.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: fsmap 1 up:standby 2024-04-26T18:20:20.501 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:20 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2972541724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6832/1924785212,v1:172.21.15.170:6833/1924785212] up:boot 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: fsmap 1 up:standby 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:20 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2972541724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6832/1924785212,v1:172.21.15.170:6833/1924785212] up:boot 2024-04-26T18:20:20.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: fsmap 1 up:standby 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:20 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2972541724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:21.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:21 smithi120 ceph-mon[32244]: Deploying daemon mds.k on smithi120 2024-04-26T18:20:21.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:21 smithi037 ceph-mon[29171]: Deploying daemon mds.k on smithi120 2024-04-26T18:20:21.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:21 smithi170 ceph-mon[32697]: Deploying daemon mds.k on smithi120 2024-04-26T18:20:22.253 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:20:22 smithi120 systemd[1]: Started Ceph mds.k for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:22.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:22.611 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:22.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:22 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:22.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:22.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:22 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:22.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:22.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:22.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:22 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:23.750 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:23 smithi037 ceph-mon[29171]: Deploying daemon mds.j on smithi037 2024-04-26T18:20:23.751 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:23 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6832/1778061281,v1:172.21.15.120:6833/1778061281] up:boot 2024-04-26T18:20:23.751 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:23 smithi037 ceph-mon[29171]: fsmap 2 up:standby 2024-04-26T18:20:23.751 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:23 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:20:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:23 smithi120 ceph-mon[32244]: Deploying daemon mds.j on smithi037 2024-04-26T18:20:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:23 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6832/1778061281,v1:172.21.15.120:6833/1778061281] up:boot 2024-04-26T18:20:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:23 smithi120 ceph-mon[32244]: fsmap 2 up:standby 2024-04-26T18:20:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:23 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:20:23.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:23 smithi170 ceph-mon[32697]: Deploying daemon mds.j on smithi037 2024-04-26T18:20:23.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:23 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6832/1778061281,v1:172.21.15.120:6833/1778061281] up:boot 2024-04-26T18:20:23.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:23 smithi170 ceph-mon[32697]: fsmap 2 up:standby 2024-04-26T18:20:23.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:23 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:20:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:24 smithi037 ceph-mon[29171]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:24.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:24 smithi120 ceph-mon[32244]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:24.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:24 smithi170 ceph-mon[32697]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2996201452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc103470 msgr2=0x7f0efc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2996201452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc103470 0x7f0efc1038f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009990 tx=0x7f0eec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2996201452 shutdown_connections 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2996201452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103e30 0x7f0efc110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2996201452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc1094c0 0x7f0efc1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2996201452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc103470 0x7f0efc1038f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.946+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2996201452 >> 172.21.15.37:0/2996201452 conn(0x7f0efc07a580 msgr2=0x7f0efc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:24.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.947+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2996201452 shutdown_connections 2024-04-26T18:20:24.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.947+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2996201452 wait complete. 2024-04-26T18:20:24.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.947+0000 7f0f03cf9700 1 Processor -- start 2024-04-26T18:20:24.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.947+0000 7f0f03cf9700 1 -- start start 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 0x7f0efc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 0x7f0efc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113610 con 0x7f0efc1094c0 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113490 con 0x7f0efc103e30 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f03cf9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113790 con 0x7f0efc103470 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43662/0 (socket says 172.21.15.37:43662) 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 -- 172.21.15.37:0/2503121015 learned_addr learned my addr 172.21.15.37:0/2503121015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f01294700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 0x7f0efc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f01a95700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 0x7f0efc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:24.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 -- 172.21.15.37:0/2503121015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 msgr2=0x7f0efc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 0x7f0efc19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 -- 172.21.15.37:0/2503121015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 msgr2=0x7f0efc19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 0x7f0efc19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 -- 172.21.15.37:0/2503121015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc19a110 con 0x7f0efc1094c0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f01a95700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 0x7f0efc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f01294700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 0x7f0efc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.948+0000 7f0f02296700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4007d70 tx=0x7f0ef400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.949+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4016020 con 0x7f0efc1094c0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.949+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef400ad40 con 0x7f0efc1094c0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.949+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc19a400 con 0x7f0efc1094c0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.949+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40115e0 con 0x7f0efc1094c0 2024-04-26T18:20:24.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.949+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1a66c0 con 0x7f0efc1094c0 2024-04-26T18:20:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.950+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc06a110 con 0x7f0efc1094c0 2024-04-26T18:20:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.951+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ef4019020 con 0x7f0efc1094c0 2024-04-26T18:20:24.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.951+0000 7f0ef2ffd700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 0x7f0ee80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.952+0000 7f0f01a95700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 0x7f0ee80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.952+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0ef40d0350 con 0x7f0efc1094c0 2024-04-26T18:20:24.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.952+0000 7f0f01a95700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 0x7f0ee80a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8005e10 tx=0x7f0ef8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:24.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:24.955+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ef4020020 con 0x7f0efc1094c0 2024-04-26T18:20:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.218+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0efc19aee0 con 0x7f0efc1094c0 2024-04-26T18:20:25.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.218+0000 7f0ef2ffd700 1 -- 172.21.15.37:0/2503121015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f0ef409d1d0 con 0x7f0efc1094c0 2024-04-26T18:20:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 msgr2=0x7f0ee80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 0x7f0ee80a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8005e10 tx=0x7f0ef8005d20 comp rx=0 tx=0).stop 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 msgr2=0x7f0efc1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4007d70 tx=0x7f0ef400a340 comp rx=0 tx=0).stop 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 shutdown_connections 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0ee80a3cc0 0x7f0ee80a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0efc1094c0 0x7f0efc1999b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0efc103470 0x7f0efc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 --2- 172.21.15.37:0/2503121015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0efc103e30 0x7f0efc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 >> 172.21.15.37:0/2503121015 conn(0x7f0efc07a580 msgr2=0x7f0efc102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 shutdown_connections 2024-04-26T18:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:25.221+0000 7f0f03cf9700 1 -- 172.21.15.37:0/2503121015 wait complete. 2024-04-26T18:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-26T18:20:25.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:25 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2503121015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:25.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:25 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2503121015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:25.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:25 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2503121015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:26.232 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 systemd[1]: Starting Ceph mds.j for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:26.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:26 smithi037 ceph-mon[29171]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:26.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:26 smithi120 ceph-mon[32244]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:26.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:26 smithi170 ceph-mon[32697]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:26.850 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 podman[66771]: 2024-04-26 18:20:26.488981799 +0000 UTC m=+0.018327366 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:26.850 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 podman[66771]: 2024-04-26 18:20:26.634023176 +0000 UTC m=+0.163368743 container create 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:26.850 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 podman[66771]: 2024-04-26 18:20:26.850289633 +0000 UTC m=+0.379635199 container init 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T18:20:27.232 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 podman[66771]: 2024-04-26 18:20:26.852873523 +0000 UTC m=+0.382219092 container start 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:20:27.233 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 ceph-mds[66790]: -- 172.21.15.37:0/3736151047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55866cb97a40 con 0x55866cb6ac00 2024-04-26T18:20:27.233 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j[66786]: starting mds.j at 2024-04-26T18:20:27.233 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 bash[66771]: 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 2024-04-26T18:20:27.233 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:20:26 smithi037 systemd[1]: Started Ceph mds.j for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: Deploying daemon mds.d on smithi037 2024-04-26T18:20:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.37:6834/1294150322,v1:172.21.15.37:6835/1294150322] up:boot 2024-04-26T18:20:28.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: fsmap 3 up:standby 2024-04-26T18:20:28.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:28 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:20:28.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:28.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: Deploying daemon mds.d on smithi037 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.37:6834/1294150322,v1:172.21.15.37:6835/1294150322] up:boot 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: fsmap 3 up:standby 2024-04-26T18:20:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:28 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:20:28.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:27 smithi037 ceph-mon[29171]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:28.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: Deploying daemon mds.d on smithi037 2024-04-26T18:20:28.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.37:6834/1294150322,v1:172.21.15.37:6835/1294150322] up:boot 2024-04-26T18:20:28.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: fsmap 3 up:standby 2024-04-26T18:20:28.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:28 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:20:28.927 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:30.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:30 smithi120 ceph-mon[32244]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:30.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:30 smithi170 ceph-mon[32697]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:30.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:30 smithi037 ceph-mon[29171]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 -- 172.21.15.37:0/2236091621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 msgr2=0x7f6b1006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/2236091621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b1006b590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009990 tx=0x7f6b00009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 -- 172.21.15.37:0/2236091621 shutdown_connections 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/2236091621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1006bad0 0x7f6b101102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/2236091621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b101094a0 0x7f6b10109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/2236091621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b1006b590 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 -- 172.21.15.37:0/2236091621 >> 172.21.15.37:0/2236091621 conn(0x7f6b100792d0 msgr2=0x7f6b100796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 -- 172.21.15.37:0/2236091621 shutdown_connections 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.715+0000 7f6b17f5d700 1 -- 172.21.15.37:0/2236091621 wait complete. 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 Processor -- start 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 -- start start 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b1006bad0 0x7f6b10102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b101094a0 0x7f6b10102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b101135b0 con 0x7f6b1006b130 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10113430 con 0x7f6b1006bad0 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b17f5d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10113730 con 0x7f6b101094a0 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b15cf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b15cf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43704/0 (socket says 172.21.15.37:43704) 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b15cf9700 1 -- 172.21.15.37:0/3122617051 learned_addr learned my addr 172.21.15.37:0/3122617051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b164fa700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b101094a0 0x7f6b10102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.716+0000 7f6b154f8700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b1006bad0 0x7f6b10102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 -- 172.21.15.37:0/3122617051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b101094a0 msgr2=0x7f6b10102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b101094a0 0x7f6b10102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 -- 172.21.15.37:0/3122617051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b1006bad0 msgr2=0x7f6b10102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b1006bad0 0x7f6b10102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 -- 172.21.15.37:0/3122617051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b101a7f70 con 0x7f6b1006b130 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b15cf9700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007e20 tx=0x7f6b0c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c016020 con 0x7f6b1006b130 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b0c00ac90 con 0x7f6b1006b130 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0115e0 con 0x7f6b1006b130 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b101a8260 con 0x7f6b1006b130 2024-04-26T18:20:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.717+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101a8720 con 0x7f6b1006b130 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.718+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10104b50 con 0x7f6b1006b130 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.719+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6b0c019020 con 0x7f6b1006b130 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.720+0000 7f6b06ffd700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 0x7f6afc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.720+0000 7f6b154f8700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 0x7f6afc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.720+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6b0c0cffb0 con 0x7f6b1006b130 2024-04-26T18:20:30.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.720+0000 7f6b154f8700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 0x7f6afc0a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b5c0 tx=0x7f6b00005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:30.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.724+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b0c020070 con 0x7f6b1006b130 2024-04-26T18:20:30.982 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:30 smithi037 podman[67219]: 2024-04-26 18:20:30.927666962 +0000 UTC m=+0.129546499 container create a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:20:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.999+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1002e490 con 0x7f6b1006b130 2024-04-26T18:20:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:30.999+0000 7f6b06ffd700 1 -- 172.21.15.37:0/3122617051 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f6b0c09cd90 con 0x7f6b1006b130 2024-04-26T18:20:31.000 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:31.000 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.001+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 msgr2=0x7f6afc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.001+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 0x7f6afc0a6180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b5c0 tx=0x7f6b00005960 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.001+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 msgr2=0x7f6b101023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.001+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c007e20 tx=0x7f6b0c00a340 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 shutdown_connections 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6afc0a3cc0 0x7f6afc0a6180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b101094a0 0x7f6b10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b1006b130 0x7f6b101023f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 --2- 172.21.15.37:0/3122617051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b1006bad0 0x7f6b10102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 >> 172.21.15.37:0/3122617051 conn(0x7f6b100792d0 msgr2=0x7f6b10100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 shutdown_connections 2024-04-26T18:20:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:31.002+0000 7f6b17f5d700 1 -- 172.21.15.37:0/3122617051 wait complete. 2024-04-26T18:20:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-04-26T18:20:31.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:31 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3122617051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:31.356 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:31 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3122617051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:31 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3122617051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:31.608 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 podman[67219]: 2024-04-26 18:20:31.357102265 +0000 UTC m=+0.558981810 container init a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:20:31.609 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 podman[67219]: 2024-04-26 18:20:31.360079841 +0000 UTC m=+0.561959379 container start a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:20:31.609 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 ceph-mds[67254]: -- 172.21.15.37:0/929434978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db76da7a40 con 0x55db76d7ac00 2024-04-26T18:20:31.609 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d[67250]: starting mds.d at 2024-04-26T18:20:31.609 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 bash[67219]: a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 2024-04-26T18:20:31.609 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:20:31 smithi037 systemd[1]: Started Ceph mds.d for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.37:6836/1073340191,v1:172.21.15.37:6837/1073340191] up:boot 2024-04-26T18:20:32.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: fsmap 4 up:standby 2024-04-26T18:20:32.754 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:32 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:20:32.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.37:6836/1073340191,v1:172.21.15.37:6837/1073340191] up:boot 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: fsmap 4 up:standby 2024-04-26T18:20:32.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:32 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.37:6836/1073340191,v1:172.21.15.37:6837/1073340191] up:boot 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: fsmap 4 up:standby 2024-04-26T18:20:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:32 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:20:33.705 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:33 smithi120 ceph-mon[32244]: Deploying daemon mds.e on smithi120 2024-04-26T18:20:33.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:33 smithi170 ceph-mon[32697]: Deploying daemon mds.e on smithi120 2024-04-26T18:20:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:33 smithi037 ceph-mon[29171]: Deploying daemon mds.e on smithi120 2024-04-26T18:20:34.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:34.252 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:20:34 smithi120 systemd[1]: Started Ceph mds.e for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:34.320 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:34.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:34.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:34 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:34.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:34 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:34.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:34.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:34.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:34.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:34.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:34 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:35.729 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:35 smithi037 ceph-mon[29171]: Deploying daemon mds.f on smithi170 2024-04-26T18:20:35.729 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:35 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6834/3605086601,v1:172.21.15.120:6835/3605086601] up:boot 2024-04-26T18:20:35.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:35 smithi037 ceph-mon[29171]: fsmap 5 up:standby 2024-04-26T18:20:35.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:35 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:20:35.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:35 smithi037 ceph-mon[29171]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:35.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:35 smithi170 ceph-mon[32697]: Deploying daemon mds.f on smithi170 2024-04-26T18:20:35.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:35 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6834/3605086601,v1:172.21.15.120:6835/3605086601] up:boot 2024-04-26T18:20:35.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:35 smithi170 ceph-mon[32697]: fsmap 5 up:standby 2024-04-26T18:20:35.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:35 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:20:35.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:35 smithi170 ceph-mon[32697]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 -- 172.21.15.37:0/3915580016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4102ce0 msgr2=0x7fd8f410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3915580016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4102ce0 0x7fd8f410f1d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b0a0 tx=0x7fd8ec00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 -- 172.21.15.37:0/3915580016 shutdown_connections 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3915580016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4102ce0 0x7fd8f410f1d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3915580016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4108390 0x7fd8f4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3915580016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 0x7fd8f41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 -- 172.21.15.37:0/3915580016 >> 172.21.15.37:0/3915580016 conn(0x7fd8f40792d0 msgr2=0x7fd8f40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 -- 172.21.15.37:0/3915580016 shutdown_connections 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.973+0000 7fd8f888c700 1 -- 172.21.15.37:0/3915580016 wait complete. 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 Processor -- start 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 -- start start 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 0x7fd8f419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 0x7fd8f419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4112500 con 0x7fd8f4108390 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4112380 con 0x7fd8f4102320 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.974+0000 7fd8f888c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4112680 con 0x7fd8f4102ce0 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52458/0 (socket says 172.21.15.37:52458) 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 -- 172.21.15.37:0/3227344716 learned_addr learned my addr 172.21.15.37:0/3227344716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 0x7fd8f419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f259c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 0x7fd8f419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 -- 172.21.15.37:0/3227344716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 msgr2=0x7fd8f419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 0x7fd8f419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 -- 172.21.15.37:0/3227344716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 msgr2=0x7fd8f419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 0x7fd8f419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 -- 172.21.15.37:0/3227344716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f419a020 con 0x7fd8f4108390 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f1d9b700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 0x7fd8f419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:20:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f2d9d700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b070 tx=0x7fd8ec004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec021070 con 0x7fd8f4108390 2024-04-26T18:20:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f419a2b0 con 0x7fd8f4108390 2024-04-26T18:20:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8ec007e30 con 0x7fd8f4108390 2024-04-26T18:20:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0046d0 con 0x7fd8f4108390 2024-04-26T18:20:35.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.975+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41a65d0 con 0x7fd8f4108390 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.976+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f406a110 con 0x7fd8f4108390 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.977+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd8ec007980 con 0x7fd8f4108390 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.978+0000 7fd8e37fe700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 0x7fd8dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.978+0000 7fd8f259c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 0x7fd8dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.978+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd8ec00f020 con 0x7fd8f4108390 2024-04-26T18:20:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.979+0000 7fd8f259c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 0x7fd8dc0a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd8f419aa00 tx=0x7fd8e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:35.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:35.983+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8ec02a020 con 0x7fd8f4108390 2024-04-26T18:20:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:35 smithi120 ceph-mon[32244]: Deploying daemon mds.f on smithi170 2024-04-26T18:20:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:35 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6834/3605086601,v1:172.21.15.120:6835/3605086601] up:boot 2024-04-26T18:20:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:35 smithi120 ceph-mon[32244]: fsmap 5 up:standby 2024-04-26T18:20:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:35 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:20:36.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:35 smithi120 ceph-mon[32244]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:36.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.238+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8f4064030 con 0x7fd8f4108390 2024-04-26T18:20:36.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.238+0000 7fd8e37fe700 1 -- 172.21.15.37:0/3227344716 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fd8ec09e230 con 0x7fd8f4108390 2024-04-26T18:20:36.239 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:36.239 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.240+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 msgr2=0x7fd8dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.240+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 0x7fd8dc0a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd8f419aa00 tx=0x7fd8e4009260 comp rx=0 tx=0).stop 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.240+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 msgr2=0x7fd8f41998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.240+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b070 tx=0x7fd8ec004cf0 comp rx=0 tx=0).stop 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 shutdown_connections 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd8dc0a3c40 0x7fd8dc0a6100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f4108390 0x7fd8f41998c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:36.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f4102320 0x7fd8f419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 --2- 172.21.15.37:0/3227344716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f4102ce0 0x7fd8f419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 >> 172.21.15.37:0/3227344716 conn(0x7fd8f40792d0 msgr2=0x7fd8f4101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 shutdown_connections 2024-04-26T18:20:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:36.241+0000 7fd8f888c700 1 -- 172.21.15.37:0/3227344716 wait complete. 2024-04-26T18:20:36.242 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-04-26T18:20:36.499 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 systemd[1]: Starting Ceph mds.f for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:36.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:36 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3227344716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:36.750 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 podman[62671]: 2024-04-26 18:20:36.401955391 +0000 UTC m=+0.018093578 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:36.750 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 podman[62671]: 2024-04-26 18:20:36.537393306 +0000 UTC m=+0.153531464 container create b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:20:36.750 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 podman[62671]: 2024-04-26 18:20:36.748232702 +0000 UTC m=+0.364370868 container init b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:20:36.750 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 podman[62671]: 2024-04-26 18:20:36.750924524 +0000 UTC m=+0.367062683 container start b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:20:36.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:36 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3227344716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:37.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:36 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3227344716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:37.075 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 ceph-mds[62690]: -- 172.21.15.170:0/3887849381 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631624afa40 con 0x563162483000 2024-04-26T18:20:37.075 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: starting mds.f at 2024-04-26T18:20:37.075 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 bash[62671]: b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 2024-04-26T18:20:37.075 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:20:36 smithi170 systemd[1]: Started Ceph mds.f for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: Deploying daemon mds.c on smithi170 2024-04-26T18:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6834/707687219,v1:172.21.15.170:6835/707687219] up:boot 2024-04-26T18:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: fsmap 6 up:standby 2024-04-26T18:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:37 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:20:38.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:38.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: Deploying daemon mds.c on smithi170 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6834/707687219,v1:172.21.15.170:6835/707687219] up:boot 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: fsmap 6 up:standby 2024-04-26T18:20:38.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:37 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: Deploying daemon mds.c on smithi170 2024-04-26T18:20:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6834/707687219,v1:172.21.15.170:6835/707687219] up:boot 2024-04-26T18:20:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: fsmap 6 up:standby 2024-04-26T18:20:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:37 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:20:38.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:39.068 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 systemd[1]: Starting Ceph mds.c for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:39.325 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 podman[62974]: 2024-04-26 18:20:39.150213404 +0000 UTC m=+0.018424778 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:39.326 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 podman[62974]: 2024-04-26 18:20:39.248445616 +0000 UTC m=+0.116656977 container create bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:39.825 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 podman[62974]: 2024-04-26 18:20:39.476539201 +0000 UTC m=+0.344750570 container init bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:39.825 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 podman[62974]: 2024-04-26 18:20:39.479791761 +0000 UTC m=+0.348003128 container start bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T18:20:39.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mds[63003]: -- 172.21.15.170:0/4096187110 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622ba567a40 con 0x5622ba53a800 2024-04-26T18:20:39.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: starting mds.c at 2024-04-26T18:20:39.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 bash[62974]: bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f 2024-04-26T18:20:39.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:20:39 smithi170 systemd[1]: Started Ceph mds.c for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:39 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:40.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:40.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:40.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:40.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:39 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:40.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:40.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:40.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:39 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:40.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.853+0000 7ff33f89a700 1 -- 172.21.15.37:0/3544325279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 msgr2=0x7ff338106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.853+0000 7ff33f89a700 1 --2- 172.21.15.37:0/3544325279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff338106fa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff3280099c0 tx=0x7ff328009cd0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 -- 172.21.15.37:0/3544325279 shutdown_connections 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 --2- 172.21.15.37:0/3544325279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff33810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 --2- 172.21.15.37:0/3544325279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff338102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 --2- 172.21.15.37:0/3544325279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff338106fa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 -- 172.21.15.37:0/3544325279 >> 172.21.15.37:0/3544325279 conn(0x7ff3380793c0 msgr2=0x7ff3380797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 -- 172.21.15.37:0/3544325279 shutdown_connections 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 -- 172.21.15.37:0/3544325279 wait complete. 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 Processor -- start 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.854+0000 7ff33f89a700 1 -- start start 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:40.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff33819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff3381a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff338112510 con 0x7ff338102f50 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff338112390 con 0x7ff3381025a0 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff338112690 con 0x7ff338107680 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33ce35700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff33819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:47632/0 (socket says 172.21.15.37:47632) 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 -- 172.21.15.37:0/2125448457 learned_addr learned my addr 172.21.15.37:0/2125448457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33de37700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff3381a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 -- 172.21.15.37:0/2125448457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 msgr2=0x7ff3381a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff3381a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 -- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 msgr2=0x7ff33819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:40.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff33819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 -- 172.21.15.37:0/2125448457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381a5f70 con 0x7ff3381025a0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33de37700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff3381a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33d636700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e9a0 tx=0x7ff33400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33400cbd0 con 0x7ff3381025a0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3381a61a0 con 0x7ff3381025a0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33400cd70 con 0x7ff3381025a0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334018680 con 0x7ff3381025a0 2024-04-26T18:20:40.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.855+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3381a66b0 con 0x7ff3381025a0 2024-04-26T18:20:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.857+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33806a110 con 0x7ff3381025a0 2024-04-26T18:20:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.858+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff334010430 con 0x7ff3381025a0 2024-04-26T18:20:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.858+0000 7ff32e7fc700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 0x7ff3240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.858+0000 7ff33ce35700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 0x7ff3240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:40.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.858+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3340d0220 con 0x7ff3381025a0 2024-04-26T18:20:40.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.859+0000 7ff33ce35700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 0x7ff3240a6100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:40.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:40.863+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff33409d000 con 0x7ff3381025a0 2024-04-26T18:20:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.124+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff33802e490 con 0x7ff3381025a0 2024-04-26T18:20:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.124+0000 7ff32e7fc700 1 -- 172.21.15.37:0/2125448457 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7ff33409c900 con 0x7ff3381025a0 2024-04-26T18:20:41.125 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:41.125 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 msgr2=0x7ff3240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 0x7ff3240a6100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff328000c00 tx=0x7ff328005fb0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 msgr2=0x7ff33819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e9a0 tx=0x7ff33400ecb0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 shutdown_connections 2024-04-26T18:20:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff3240a3c40 0x7ff3240a6100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff338107680 0x7ff3381a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3381025a0 0x7ff33819a9b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 --2- 172.21.15.37:0/2125448457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff338102f50 0x7ff33819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 >> 172.21.15.37:0/2125448457 conn(0x7ff3380793c0 msgr2=0x7ff338108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 shutdown_connections 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:41.127+0000 7ff33f89a700 1 -- 172.21.15.37:0/2125448457 wait complete. 2024-04-26T18:20:41.128 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-26T18:20:41.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:40 smithi037 ceph-mon[29171]: Deploying daemon mds.h on smithi120 2024-04-26T18:20:41.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:40 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6836/2099915678,v1:172.21.15.170:6837/2099915678] up:boot 2024-04-26T18:20:41.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:40 smithi037 ceph-mon[29171]: fsmap 7 up:standby 2024-04-26T18:20:41.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:40 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:20:41.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:40 smithi120 ceph-mon[32244]: Deploying daemon mds.h on smithi120 2024-04-26T18:20:41.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:40 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6836/2099915678,v1:172.21.15.170:6837/2099915678] up:boot 2024-04-26T18:20:41.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:40 smithi120 ceph-mon[32244]: fsmap 7 up:standby 2024-04-26T18:20:41.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:40 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:20:41.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:40 smithi170 ceph-mon[32697]: Deploying daemon mds.h on smithi120 2024-04-26T18:20:41.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:40 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6836/2099915678,v1:172.21.15.170:6837/2099915678] up:boot 2024-04-26T18:20:41.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:40 smithi170 ceph-mon[32697]: fsmap 7 up:standby 2024-04-26T18:20:41.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:40 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:20:41.911 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:41 smithi120 systemd[1]: Starting Ceph mds.h for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:42.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:41 smithi120 ceph-mon[32244]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:42.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:41 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2125448457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:42.175 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 podman[62583]: 2024-04-26 18:20:42.047896425 +0000 UTC m=+0.113574832 container create f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:42.175 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 podman[62583]: 2024-04-26 18:20:41.952244613 +0000 UTC m=+0.017923016 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:42.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:41 smithi037 ceph-mon[29171]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:42.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:41 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2125448457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:42.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:41 smithi170 ceph-mon[32697]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:42.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:41 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2125448457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 podman[62583]: 2024-04-26 18:20:42.267417146 +0000 UTC m=+0.333095550 container init f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 podman[62583]: 2024-04-26 18:20:42.270181484 +0000 UTC m=+0.335859889 container start f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 ceph-mds[62602]: -- 172.21.15.120:0/2982320255 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0c1737a40 con 0x55b0c170a800 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: starting mds.h at 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 bash[62583]: f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b 2024-04-26T18:20:42.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:20:42 smithi120 systemd[1]: Started Ceph mds.h for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: Deploying daemon mds.i on smithi170 2024-04-26T18:20:43.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:boot 2024-04-26T18:20:43.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: fsmap 8 up:standby 2024-04-26T18:20:43.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:43 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: Deploying daemon mds.i on smithi170 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:boot 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: fsmap 8 up:standby 2024-04-26T18:20:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:43 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:43.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: Deploying daemon mds.i on smithi170 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:boot 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: fsmap 8 up:standby 2024-04-26T18:20:43.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:43 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:43.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:44.129 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:44.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:44 smithi037 ceph-mon[29171]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:44 smithi170 ceph-mon[32697]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:44.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:44 smithi120 ceph-mon[32244]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:44.880 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:20:44 smithi170 podman[63292]: 2024-04-26 18:20:44.857295109 +0000 UTC m=+0.362699161 container start 1bdd765c1791863984adc2f6875c6f3e910637699361607b1364944e6ae944c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:20:44.880 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:20:44 smithi170 ceph-mds[63311]: -- 172.21.15.170:0/3903310873 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d93a3e5a40 con 0x55d93a3b8c00 2024-04-26T18:20:45.325 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:20:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: starting mds.i at 2024-04-26T18:20:45.325 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:20:44 smithi170 bash[63292]: 1bdd765c1791863984adc2f6875c6f3e910637699361607b1364944e6ae944c1 2024-04-26T18:20:45.326 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:20:44 smithi170 systemd[1]: Started Ceph mds.i for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.866+0000 7f26cab36700 1 -- 172.21.15.37:0/3001921369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103470 msgr2=0x7f26c41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.866+0000 7f26cab36700 1 --2- 172.21.15.37:0/3001921369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103470 0x7f26c41038f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009990 tx=0x7f26b0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.866+0000 7f26cab36700 1 -- 172.21.15.37:0/3001921369 shutdown_connections 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.866+0000 7f26cab36700 1 --2- 172.21.15.37:0/3001921369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103e30 0x7f26c4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 --2- 172.21.15.37:0/3001921369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 0x7f26c41098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 --2- 172.21.15.37:0/3001921369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103470 0x7f26c41038f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 -- 172.21.15.37:0/3001921369 >> 172.21.15.37:0/3001921369 conn(0x7f26c407a580 msgr2=0x7f26c407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 -- 172.21.15.37:0/3001921369 shutdown_connections 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 -- 172.21.15.37:0/3001921369 wait complete. 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 Processor -- start 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.867+0000 7f26cab36700 1 -- start start 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 0x7f26c4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 0x7f26c40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26c4113660 con 0x7f26c4103e30 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41134e0 con 0x7f26c4103470 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26cab36700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26c41137e0 con 0x7f26c41094c0 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:45.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48214/0 (socket says 172.21.15.37:48214) 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 -- 172.21.15.37:0/3685489083 learned_addr learned my addr 172.21.15.37:0/3685489083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c90d3700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 0x7f26c40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c88d2700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 0x7f26c4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 -- 172.21.15.37:0/3685489083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 msgr2=0x7f26c40761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 0x7f26c40761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 -- 172.21.15.37:0/3685489083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 msgr2=0x7f26c4075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 0x7f26c4075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c3fff700 1 -- 172.21.15.37:0/3685489083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c41a80e0 con 0x7f26c4103e30 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.868+0000 7f26c88d2700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 0x7f26c4075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26c3fff700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0027070 con 0x7f26c4103e30 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b0003f20 con 0x7f26c4103e30 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c41a8370 con 0x7f26c4103e30 2024-04-26T18:20:45.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0022cf0 con 0x7f26c4103e30 2024-04-26T18:20:45.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.869+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c41a88a0 con 0x7f26c4103e30 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.870+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c406a110 con 0x7f26c4103e30 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.871+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26b002a070 con 0x7f26c4103e30 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.872+0000 7f26c1ffb700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 0x7f26b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.872+0000 7f26c88d2700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 0x7f26b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.872+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f26b00d0ca0 con 0x7f26c4103e30 2024-04-26T18:20:45.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.872+0000 7f26c88d2700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 0x7f26b40a6180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b800bcf0 tx=0x7f26b800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:45.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:45.876+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26b0013070 con 0x7f26c4103e30 2024-04-26T18:20:46.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.141+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26c4076c50 con 0x7f26c4103e30 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: Deploying daemon mds.g on smithi037 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:boot 2024-04-26T18:20:46.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: fsmap 9 up:standby 2024-04-26T18:20:46.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:46 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:46.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.142+0000 7f26c1ffb700 1 -- 172.21.15.37:0/3685489083 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f26b009dbb0 con 0x7f26c4103e30 2024-04-26T18:20:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:46.143 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.144+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 msgr2=0x7f26b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.144+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 0x7f26b40a6180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b800bcf0 tx=0x7f26b800b2c0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.144+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 msgr2=0x7f26c4075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.144+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00047b0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 shutdown_connections 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f26b40a3cc0 0x7f26b40a6180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26c41094c0 0x7f26c40761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26c4103470 0x7f26c4075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 --2- 172.21.15.37:0/3685489083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26c4103e30 0x7f26c4075c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 >> 172.21.15.37:0/3685489083 conn(0x7f26c407a580 msgr2=0x7f26c41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 shutdown_connections 2024-04-26T18:20:46.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:46.145+0000 7f26cab36700 1 -- 172.21.15.37:0/3685489083 wait complete. 2024-04-26T18:20:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-04-26T18:20:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: Deploying daemon mds.g on smithi037 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:boot 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: fsmap 9 up:standby 2024-04-26T18:20:46.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:46 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: Deploying daemon mds.g on smithi037 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:boot 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: fsmap 9 up:standby 2024-04-26T18:20:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:46 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:47.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:47 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3685489083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:47.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:47 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3685489083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:47.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:47 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3685489083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:48.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:48 smithi170 ceph-mon[32697]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:48.444 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:48 smithi037 ceph-mon[29171]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:48.445 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 systemd[1]: Starting Ceph mds.g for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:48 smithi120 ceph-mon[32244]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:48.712 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 podman[68025]: 2024-04-26 18:20:48.347822928 +0000 UTC m=+0.018434722 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:48.713 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 podman[68025]: 2024-04-26 18:20:48.459729171 +0000 UTC m=+0.130341005 container create 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:20:48.713 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 podman[68025]: 2024-04-26 18:20:48.687082834 +0000 UTC m=+0.357694616 container init 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:20:48.713 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 podman[68025]: 2024-04-26 18:20:48.68993883 +0000 UTC m=+0.360550621 container start 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T18:20:48.982 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 ceph-mds[68044]: -- 172.21.15.37:0/677965760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a3ee2da40 con 0x564a3ee01000 2024-04-26T18:20:48.983 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g[68040]: starting mds.g at 2024-04-26T18:20:48.983 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 bash[68025]: 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 2024-04-26T18:20:48.983 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:20:48 smithi037 systemd[1]: Started Ceph mds.g for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:49.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:49.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: Deploying daemon mds.a on smithi037 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.37:6838/1645821272,v1:172.21.15.37:6839/1645821272] up:boot 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: fsmap 10 up:standby 2024-04-26T18:20:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:49 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:20:50.089 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: Deploying daemon mds.a on smithi037 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.37:6838/1645821272,v1:172.21.15.37:6839/1645821272] up:boot 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: fsmap 10 up:standby 2024-04-26T18:20:50.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:49 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:20:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:50.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: Deploying daemon mds.a on smithi037 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.37:6838/1645821272,v1:172.21.15.37:6839/1645821272] up:boot 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: fsmap 10 up:standby 2024-04-26T18:20:50.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:49 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:20:51.233 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:51 smithi037 systemd[1]: Starting Ceph mds.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:52.093 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:51 smithi037 ceph-mon[29171]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:52.094 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:51 smithi037 podman[68437]: 2024-04-26 18:20:51.828467626 +0000 UTC m=+0.024394886 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:52.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:51 smithi120 ceph-mon[32244]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:52.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:51 smithi170 ceph-mon[32697]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:52.465 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 podman[68437]: 2024-04-26 18:20:52.197458335 +0000 UTC m=+0.393385574 container create f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 -- 172.21.15.37:0/3975249802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006b130 msgr2=0x7ff4d006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 --2- 172.21.15.37:0/3975249802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006b130 0x7ff4d006b590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009990 tx=0x7ff4c4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 -- 172.21.15.37:0/3975249802 shutdown_connections 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 --2- 172.21.15.37:0/3975249802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006bad0 0x7ff4d01102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 --2- 172.21.15.37:0/3975249802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 0x7ff4d0109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 --2- 172.21.15.37:0/3975249802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006b130 0x7ff4d006b590 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 -- 172.21.15.37:0/3975249802 >> 172.21.15.37:0/3975249802 conn(0x7ff4d00792d0 msgr2=0x7ff4d00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 -- 172.21.15.37:0/3975249802 shutdown_connections 2024-04-26T18:20:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.571+0000 7ff4d6862700 1 -- 172.21.15.37:0/3975249802 wait complete. 2024-04-26T18:20:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 Processor -- start 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 -- start start 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 0x7ff4d01023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 0x7ff4d0102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01135d0 con 0x7ff4d006bad0 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0113450 con 0x7ff4d01094a0 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.572+0000 7ff4d6862700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0113750 con 0x7ff4d006b130 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48252/0 (socket says 172.21.15.37:48252) 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 -- 172.21.15.37:0/861376022 learned_addr learned my addr 172.21.15.37:0/861376022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4d4dff700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 0x7ff4d0102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cffff700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 0x7ff4d01023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 -- 172.21.15.37:0/861376022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 msgr2=0x7ff4d01023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 0x7ff4d01023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 -- 172.21.15.37:0/861376022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 msgr2=0x7ff4d0102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 0x7ff4d0102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 -- 172.21.15.37:0/861376022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d01ac4b0 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cffff700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 0x7ff4d01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cf7fe700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4027070 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d01ac6e0 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c40040e0 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400f460 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.573+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d01acb90 con 0x7ff4d006bad0 2024-04-26T18:20:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.575+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d0104b50 con 0x7ff4d006bad0 2024-04-26T18:20:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.575+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4c402a070 con 0x7ff4d006bad0 2024-04-26T18:20:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.576+0000 7ff4cd7fa700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 0x7ff4b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.576+0000 7ff4cffff700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 0x7ff4b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.576+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4c40d0f20 con 0x7ff4d006bad0 2024-04-26T18:20:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.577+0000 7ff4cffff700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 0x7ff4b80a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005fd0 tx=0x7ff4c0005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:52.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.581+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4c405eff0 con 0x7ff4d006bad0 2024-04-26T18:20:52.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.858+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d006a110 con 0x7ff4d006bad0 2024-04-26T18:20:52.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.859+0000 7ff4cd7fa700 1 -- 172.21.15.37:0/861376022 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7ff4c4013070 con 0x7ff4d006bad0 2024-04-26T18:20:52.859 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:52.859 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T18:20:52.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 msgr2=0x7ff4b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 0x7ff4b80a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005fd0 tx=0x7ff4c0005db0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 msgr2=0x7ff4d0102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4000c00 tx=0x7ff4c40047b0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 shutdown_connections 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff4b80a3c40 0x7ff4b80a6100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4d01094a0 0x7ff4d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006b130 0x7ff4d01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 --2- 172.21.15.37:0/861376022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4d006bad0 0x7ff4d0102930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 >> 172.21.15.37:0/861376022 conn(0x7ff4d00792d0 msgr2=0x7ff4d0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 shutdown_connections 2024-04-26T18:20:52.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:52.861+0000 7ff4d6862700 1 -- 172.21.15.37:0/861376022 wait complete. 2024-04-26T18:20:52.863 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 podman[68437]: 2024-04-26 18:20:52.773185028 +0000 UTC m=+0.969112268 container init f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 podman[68437]: 2024-04-26 18:20:52.776723812 +0000 UTC m=+0.972651051 container start f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 ceph-mds[68489]: -- 172.21.15.37:0/2877394936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a66d37a40 con 0x562a66d0ac00 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a[68463]: starting mds.a at 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 bash[68437]: f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 2024-04-26T18:20:53.229 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:20:52 smithi037 systemd[1]: Started Ceph mds.a for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.37:6840/3026662936,v1:172.21.15.37:6841/3026662936] up:boot 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: fsmap 11 up:standby 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/861376022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:53 smithi037 ceph-mon[29171]: Deploying daemon mds.b on smithi120 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.37:6840/3026662936,v1:172.21.15.37:6841/3026662936] up:boot 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: fsmap 11 up:standby 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/861376022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:54.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:53 smithi120 ceph-mon[32244]: Deploying daemon mds.b on smithi120 2024-04-26T18:20:54.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:54.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.37:6840/3026662936,v1:172.21.15.37:6841/3026662936] up:boot 2024-04-26T18:20:54.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: fsmap 11 up:standby 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/861376022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:54.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:53 smithi170 ceph-mon[32697]: Deploying daemon mds.b on smithi120 2024-04-26T18:20:55.253 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 systemd[1]: Starting Ceph mds.b for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:20:55.530 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 podman[62893]: 2024-04-26 18:20:55.269474988 +0000 UTC m=+0.094691365 container create c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:55.530 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 podman[62893]: 2024-04-26 18:20:55.192568187 +0000 UTC m=+0.017784538 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:55.530 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 podman[62893]: 2024-04-26 18:20:55.505411354 +0000 UTC m=+0.330627686 container init c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:20:55.530 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 podman[62893]: 2024-04-26 18:20:55.507973014 +0000 UTC m=+0.333189349 container start c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T18:20:55.530 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mds[62913]: -- 172.21.15.120:0/3214938187 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cec363a40 con 0x558cec337000 2024-04-26T18:20:55.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:20:55.829 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:55.954 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: starting mds.b at 2024-04-26T18:20:55.954 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 bash[62893]: c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c 2024-04-26T18:20:55.955 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:20:55 smithi120 systemd[1]: Started Ceph mds.b for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:20:56.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:56.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:55 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:56.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:56.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:55 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:56.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:55 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:56 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:boot 2024-04-26T18:20:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:56 smithi037 ceph-mon[29171]: fsmap 12 up:standby 2024-04-26T18:20:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:20:57.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:56 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:57.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:56 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:boot 2024-04-26T18:20:57.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:56 smithi120 ceph-mon[32244]: fsmap 12 up:standby 2024-04-26T18:20:57.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:20:57.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:56 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:56 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:boot 2024-04-26T18:20:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:56 smithi170 ceph-mon[32697]: fsmap 12 up:standby 2024-04-26T18:20:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:20:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:56 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 -- 172.21.15.37:0/3551549668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0103310 msgr2=0x7f5cc0106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3551549668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0103310 0x7f5cc0106f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009990 tx=0x7f5cb4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 -- 172.21.15.37:0/3551549668 shutdown_connections 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3551549668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc01076b0 0x7f5cc010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3551549668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0102960 0x7f5cc0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3551549668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0103310 0x7f5cc0106f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.490+0000 7f5cc8620700 1 -- 172.21.15.37:0/3551549668 >> 172.21.15.37:0/3551549668 conn(0x7f5cc00792d0 msgr2=0x7f5cc00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 -- 172.21.15.37:0/3551549668 shutdown_connections 2024-04-26T18:20:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 -- 172.21.15.37:0/3551549668 wait complete. 2024-04-26T18:20:57.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 Processor -- start 2024-04-26T18:20:57.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 -- start start 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102960 0x7f5cc019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.491+0000 7f5cc8620700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 0x7f5cc019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc8620700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc8620700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112500 con 0x7f5cc01076b0 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc8620700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112380 con 0x7f5cc0102960 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc8620700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0112680 con 0x7f5cc0103310 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc63bc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102960 0x7f5cc019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc5bbb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 0x7f5cc019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49734/0 (socket says 172.21.15.37:49734) 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 -- 172.21.15.37:0/3489124211 learned_addr learned my addr 172.21.15.37:0/3489124211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 -- 172.21.15.37:0/3489124211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 msgr2=0x7f5cc019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:57.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 0x7f5cc019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 -- 172.21.15.37:0/3489124211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102960 msgr2=0x7f5cc019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102960 0x7f5cc019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 -- 172.21.15.37:0/3489124211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc019a0d0 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc5bbb700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 0x7f5cc019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cc6bbd700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e940 tx=0x7f5cb000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0004a30 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.492+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb00054d0 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.493+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc019a3c0 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.493+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000ccf0 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.493+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc0103f50 con 0x7f5cc01076b0 2024-04-26T18:20:57.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.494+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc006b3a0 con 0x7f5cc01076b0 2024-04-26T18:20:57.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.495+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5cb0005020 con 0x7f5cc01076b0 2024-04-26T18:20:57.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.495+0000 7f5cbf7fe700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 0x7f5ca80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:57.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.496+0000 7f5cc63bc700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 0x7f5ca80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:57.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.496+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5cb00d00d0 con 0x7f5cc01076b0 2024-04-26T18:20:57.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.496+0000 7f5cc63bc700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 0x7f5ca80a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cc019aab0 tx=0x7f5cb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:57.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.504+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cb009ceb0 con 0x7f5cc01076b0 2024-04-26T18:20:57.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.782+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cc019aea0 con 0x7f5cc01076b0 2024-04-26T18:20:57.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.783+0000 7f5cbf7fe700 1 -- 172.21.15.37:0/3489124211 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f5cb009c7b0 con 0x7f5cc01076b0 2024-04-26T18:20:57.783 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:20:57.783 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T18:20:57.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 msgr2=0x7f5ca80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 0x7f5ca80a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cc019aab0 tx=0x7f5cb800b040 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 msgr2=0x7f5cc0199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000e940 tx=0x7f5cb000ec50 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 shutdown_connections 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5ca80a3c40 0x7f5ca80a6100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc01076b0 0x7f5cc0199970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5cc0102960 0x7f5cc019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 --2- 172.21.15.37:0/3489124211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cc0103310 0x7f5cc019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 >> 172.21.15.37:0/3489124211 conn(0x7f5cc00792d0 msgr2=0x7f5cc006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 shutdown_connections 2024-04-26T18:20:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:20:57.785+0000 7f5cc8620700 1 -- 172.21.15.37:0/3489124211 wait complete. 2024-04-26T18:20:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:57 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3489124211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:58.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:57 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3489124211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:20:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:20:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:57 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3489124211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:20:58.948 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T18:20:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:00.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:20:59 smithi037 ceph-mon[29171]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:00.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:20:59 smithi120 ceph-mon[32244]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:20:59 smithi170 ceph-mon[32697]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 -- 172.21.15.37:0/223907470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d6006b9c0 msgr2=0x7f9d6006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 --2- 172.21.15.37:0/223907470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d6006b9c0 0x7f9d6006be20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009990 tx=0x7f9d54009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 -- 172.21.15.37:0/223907470 shutdown_connections 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 --2- 172.21.15.37:0/223907470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 0x7f9d601102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 --2- 172.21.15.37:0/223907470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d60109490 0x7f9d60109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 --2- 172.21.15.37:0/223907470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d6006b9c0 0x7f9d6006be20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 -- 172.21.15.37:0/223907470 >> 172.21.15.37:0/223907470 conn(0x7f9d60079290 msgr2=0x7f9d600796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.878+0000 7f9d66f43700 1 -- 172.21.15.37:0/223907470 shutdown_connections 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 -- 172.21.15.37:0/223907470 wait complete. 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 Processor -- start 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 -- start start 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d6006b9c0 0x7f9d6019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 0x7f9d6019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601135a0 con 0x7f9d60109490 2024-04-26T18:21:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60113420 con 0x7f9d6006c360 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.879+0000 7f9d66f43700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60113720 con 0x7f9d6006b9c0 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d64cdf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d6006b9c0 0x7f9d6019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49752/0 (socket says 172.21.15.37:49752) 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 -- 172.21.15.37:0/548938518 learned_addr learned my addr 172.21.15.37:0/548938518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d5ffff700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 0x7f9d6019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 -- 172.21.15.37:0/548938518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d6006b9c0 msgr2=0x7f9d6019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d6006b9c0 0x7f9d6019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 -- 172.21.15.37:0/548938518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 msgr2=0x7f9d6019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 0x7f9d6019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 -- 172.21.15.37:0/548938518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6019a0a0 con 0x7f9d60109490 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d654e0700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800d980 tx=0x7f9d5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58017070 con 0x7f9d60109490 2024-04-26T18:21:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.880+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6019a390 con 0x7f9d60109490 2024-04-26T18:21:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.881+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d58005020 con 0x7f9d60109490 2024-04-26T18:21:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.881+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800fa10 con 0x7f9d60109490 2024-04-26T18:21:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.881+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601a6600 con 0x7f9d60109490 2024-04-26T18:21:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.882+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6006a110 con 0x7f9d60109490 2024-04-26T18:21:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.883+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d58004a30 con 0x7f9d60109490 2024-04-26T18:21:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.884+0000 7f9d5dffb700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 0x7f9d480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.884+0000 7f9d64cdf700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 0x7f9d480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.884+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9d580cfcd0 con 0x7f9d60109490 2024-04-26T18:21:00.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.885+0000 7f9d64cdf700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 0x7f9d480a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9d60108770 tx=0x7f9d50009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:00.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:00.889+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d58020020 con 0x7f9d60109490 2024-04-26T18:21:01.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.104+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9d6002e490 con 0x7f9d60109490 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.104+0000 7f9d5dffb700 1 -- 172.21.15.37:0/548938518 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f9d5809cb50 con 0x7f9d60109490 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:21:01.105 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 msgr2=0x7f9d480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 0x7f9d480a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9d60108770 tx=0x7f9d50009180 comp rx=0 tx=0).stop 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 msgr2=0x7f9d60199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800d980 tx=0x7f9d5800dc90 comp rx=0 tx=0).stop 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 shutdown_connections 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9d480a3cc0 0x7f9d480a6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d60109490 0x7f9d60199940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d6006b9c0 0x7f9d6019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:01.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 --2- 172.21.15.37:0/548938518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d6006c360 0x7f9d6019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:01.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 >> 172.21.15.37:0/548938518 conn(0x7f9d60079290 msgr2=0x7f9d60100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:01.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 shutdown_connections 2024-04-26T18:21:01.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:01.107+0000 7f9d66f43700 1 -- 172.21.15.37:0/548938518 wait complete. 2024-04-26T18:21:01.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/548938518' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:21:01.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:01 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/548938518' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:21:01.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:01 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:01.763 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T18:21:01.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T18:21:01.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:01.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:01.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:01.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/548938518' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:21:01.826 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:01 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:02 smithi170 ceph-mon[32697]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:02.663 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:02.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:02 smithi037 ceph-mon[29171]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:02.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:02 smithi120 ceph-mon[32244]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:04.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:03 smithi037 ceph-mon[29171]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:04.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:03 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:03 smithi120 ceph-mon[32244]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:04.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:03 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:03 smithi170 ceph-mon[32697]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:04.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:03 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 -- 172.21.15.37:0/3485135495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 msgr2=0x7fc24c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 --2- 172.21.15.37:0/3485135495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c1102e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc24400b0a0 tx=0x7fc24400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 -- 172.21.15.37:0/3485135495 shutdown_connections 2024-04-26T18:21:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 --2- 172.21.15.37:0/3485135495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c1102e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 --2- 172.21.15.37:0/3485135495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 0x7fc24c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 --2- 172.21.15.37:0/3485135495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 0x7fc24c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 -- 172.21.15.37:0/3485135495 >> 172.21.15.37:0/3485135495 conn(0x7fc24c0792d0 msgr2=0x7fc24c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 -- 172.21.15.37:0/3485135495 shutdown_connections 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.308+0000 7fc2546e0700 1 -- 172.21.15.37:0/3485135495 wait complete. 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 Processor -- start 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 -- start start 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 0x7fc24c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 0x7fc24c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c113620 con 0x7fc24c06bad0 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1134a0 con 0x7fc24c06b130 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc2546e0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1137a0 con 0x7fc24c1094a0 2024-04-26T18:21:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc251c7b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc251c7b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49780/0 (socket says 172.21.15.37:49780) 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc251c7b700 1 -- 172.21.15.37:0/78959528 learned_addr learned my addr 172.21.15.37:0/78959528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.309+0000 7fc25247c700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 0x7fc24c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc252c7d700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 0x7fc24c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 -- 172.21.15.37:0/78959528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 msgr2=0x7fc24c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 0x7fc24c102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 -- 172.21.15.37:0/78959528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 msgr2=0x7fc24c1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 0x7fc24c1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 -- 172.21.15.37:0/78959528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c1a8090 con 0x7fc24c06bad0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc251c7b700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c9b0 tx=0x7fc23c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c007b00 con 0x7fc24c06bad0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc23c007ca0 con 0x7fc24c06bad0 2024-04-26T18:21:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c1a8380 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c018660 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.310+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c1a88c0 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.311+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c10ba10 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.312+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc23c00f450 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.313+0000 7fc2437fe700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 0x7fc2380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.313+0000 7fc25247c700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 0x7fc2380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.313+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc23c0cffc0 con 0x7fc24c06bad0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.313+0000 7fc25247c700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 0x7fc2380a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc248009a10 tx=0x7fc248009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.317+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc23c014030 con 0x7fc24c06bad0 2024-04-26T18:21:04.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.544+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc24c06a110 con 0x7fc24c06bad0 2024-04-26T18:21:04.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.860+0000 7fc2437fe700 1 -- 172.21.15.37:0/78959528 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fc23c09cda0 con 0x7fc24c06bad0 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 msgr2=0x7fc2380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 0x7fc2380a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc248009a10 tx=0x7fc248009190 comp rx=0 tx=0).stop 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 msgr2=0x7fc24c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c9b0 tx=0x7fc23c00ccc0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 shutdown_connections 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2380a3c40 0x7fc2380a6100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c1094a0 0x7fc24c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc24c06b130 0x7fc24c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 --2- 172.21.15.37:0/78959528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc24c06bad0 0x7fc24c102930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:04.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 >> 172.21.15.37:0/78959528 conn(0x7fc24c0792d0 msgr2=0x7fc24c100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:04.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 shutdown_connections 2024-04-26T18:21:04.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:04.864+0000 7fc2546e0700 1 -- 172.21.15.37:0/78959528 wait complete. 2024-04-26T18:21:04.867 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:05.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:04 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:21:05.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:05.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:05.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:04 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:21:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:05.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:05.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:04 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:21:05.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T18:21:05.726 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:06.165 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:05 smithi037 ceph-mon[29171]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:06.165 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:05 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:21:06.165 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:05 smithi037 ceph-mon[29171]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:21:06.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:05 smithi170 ceph-mon[32697]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:06.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:05 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:21:06.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:05 smithi170 ceph-mon[32697]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:21:06.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:05 smithi120 ceph-mon[32244]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:06.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:05 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/78959528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:21:06.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:05 smithi120 ceph-mon[32244]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:21:07.075 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:06 smithi037 ceph-mon[29171]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:21:07.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:06 smithi120 ceph-mon[32244]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 -- 172.21.15.37:0/1557192232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c1094e0 msgr2=0x7f1b8c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 --2- 172.21.15.37:0/1557192232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c1098c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009990 tx=0x7f1b7c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 -- 172.21.15.37:0/1557192232 shutdown_connections 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 --2- 172.21.15.37:0/1557192232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c103e60 0x7f1b8c1102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 --2- 172.21.15.37:0/1557192232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c1098c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 --2- 172.21.15.37:0/1557192232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1034a0 0x7f1b8c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 -- 172.21.15.37:0/1557192232 >> 172.21.15.37:0/1557192232 conn(0x7f1b8c07a590 msgr2=0x7f1b8c07a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 -- 172.21.15.37:0/1557192232 shutdown_connections 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 -- 172.21.15.37:0/1557192232 wait complete. 2024-04-26T18:21:07.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.301+0000 7f1b925be700 1 Processor -- start 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 -- start start 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c1034a0 0x7f1b8c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c113660 con 0x7f1b8c103e60 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c1134e0 con 0x7f1b8c1034a0 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b925be700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c1137e0 con 0x7f1b8c1094e0 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37504/0 (socket says 172.21.15.37:37504) 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8b7fe700 1 -- 172.21.15.37:0/3583618261 learned_addr learned my addr 172.21.15.37:0/3583618261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b90b5b700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8bfff700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c1034a0 0x7f1b8c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8b7fe700 1 -- 172.21.15.37:0/3583618261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 msgr2=0x7f1b8c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.302+0000 7f1b8b7fe700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b8b7fe700 1 -- 172.21.15.37:0/3583618261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c1034a0 msgr2=0x7f1b8c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b8b7fe700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c1034a0 0x7f1b8c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b8b7fe700 1 -- 172.21.15.37:0/3583618261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c1a80b0 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b90b5b700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b8b7fe700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e9a0 tx=0x7f1b7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400ca10 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1a83a0 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7400cbb0 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74018680 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.303+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c1a8930 con 0x7f1b8c103e60 2024-04-26T18:21:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.304+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c10b950 con 0x7f1b8c103e60 2024-04-26T18:21:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.305+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b74010430 con 0x7f1b8c103e60 2024-04-26T18:21:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.306+0000 7f1b897fa700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 0x7f1b780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.306+0000 7f1b8bfff700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 0x7f1b780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.306+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f1b740d02d0 con 0x7f1b8c103e60 2024-04-26T18:21:07.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.306+0000 7f1b8bfff700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 0x7f1b780a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b2e0 tx=0x7f1b7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.311+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b7409cf40 con 0x7f1b8c103e60 2024-04-26T18:21:07.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:06 smithi170 ceph-mon[32697]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:21:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.518+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1b8c06a110 con 0x7f1b8c103e60 2024-04-26T18:21:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.881+0000 7f1b897fa700 1 -- 172.21.15.37:0/3583618261 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f1b7409c840 con 0x7f1b8c103e60 2024-04-26T18:21:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.885+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 msgr2=0x7f1b780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.885+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 0x7f1b780a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00b2e0 tx=0x7f1b7c005fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.885+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 msgr2=0x7f1b8c075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:07.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.885+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e9a0 tx=0x7f1b7400ecb0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 shutdown_connections 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b780a3c40 0x7f1b780a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b8c1094e0 0x7f1b8c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b8c1034a0 0x7f1b8c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 --2- 172.21.15.37:0/3583618261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b8c103e60 0x7f1b8c075c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.886+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 >> 172.21.15.37:0/3583618261 conn(0x7f1b8c07a590 msgr2=0x7f1b8c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:07.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.887+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 shutdown_connections 2024-04-26T18:21:07.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:07.887+0000 7f1b925be700 1 -- 172.21.15.37:0/3583618261 wait complete. 2024-04-26T18:21:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-26T18:21:08.207 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:07 smithi120 ceph-mon[32244]: pgmap v288: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:08.207 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:07 smithi120 ceph-mon[32244]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:21:08.207 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:07 smithi120 ceph-mon[32244]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:21:08.207 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:07 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:21:08.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:07 smithi037 ceph-mon[29171]: pgmap v288: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:08.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:07 smithi037 ceph-mon[29171]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:21:08.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:07 smithi037 ceph-mon[29171]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:21:08.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:07 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:21:08.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:07 smithi170 ceph-mon[32697]: pgmap v288: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:08.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:07 smithi170 ceph-mon[32697]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:21:08.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:07 smithi170 ceph-mon[32697]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:21:08.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:07 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:21:08.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T18:21:08.817 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:09.097 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:08 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:21:09.097 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:08 smithi037 ceph-mon[29171]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:21:09.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:08 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:21:09.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:08 smithi120 ceph-mon[32244]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:21:09.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:08 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3583618261' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:21:09.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:08 smithi170 ceph-mon[32697]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:21:10.154 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:09 smithi120 ceph-mon[32244]: pgmap v291: 97 pgs: 26 creating+peering, 8 active+clean, 63 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:10.154 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:09 smithi120 ceph-mon[32244]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:21:10.154 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:09 smithi120 ceph-mon[32244]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:21:10.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:09 smithi037 ceph-mon[29171]: pgmap v291: 97 pgs: 26 creating+peering, 8 active+clean, 63 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:10.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:09 smithi037 ceph-mon[29171]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:21:10.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:09 smithi037 ceph-mon[29171]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:21:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:09 smithi170 ceph-mon[32697]: pgmap v291: 97 pgs: 26 creating+peering, 8 active+clean, 63 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:09 smithi170 ceph-mon[32697]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:21:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:09 smithi170 ceph-mon[32697]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 -- 172.21.15.37:0/18534855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406c590 msgr2=0x7fadc4110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 --2- 172.21.15.37:0/18534855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406c590 0x7fadc4110310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00b0a0 tx=0x7fadbc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 -- 172.21.15.37:0/18534855 shutdown_connections 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 --2- 172.21.15.37:0/18534855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406c590 0x7fadc4110310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 --2- 172.21.15.37:0/18534855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc41094d0 0x7fadc41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 --2- 172.21.15.37:0/18534855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc406bbd0 0x7fadc406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 -- 172.21.15.37:0/18534855 >> 172.21.15.37:0/18534855 conn(0x7fadc40793c0 msgr2=0x7fadc40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 -- 172.21.15.37:0/18534855 shutdown_connections 2024-04-26T18:21:10.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 -- 172.21.15.37:0/18534855 wait complete. 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.729+0000 7fadcbca8700 1 Processor -- start 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 -- start start 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc406c590 0x7fadc41028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc41094d0 0x7fadc4102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41136b0 con 0x7fadc406bbd0 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4113530 con 0x7fadc41094d0 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadcbca8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4113830 con 0x7fadc406c590 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37526/0 (socket says 172.21.15.37:37526) 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 -- 172.21.15.37:0/1134000219 learned_addr learned my addr 172.21.15.37:0/1134000219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadca245700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc41094d0 0x7fadc4102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9243700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc406c590 0x7fadc41028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:10.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 -- 172.21.15.37:0/1134000219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc406c590 msgr2=0x7fadc41028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc406c590 0x7fadc41028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 -- 172.21.15.37:0/1134000219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc41094d0 msgr2=0x7fadc4102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc41094d0 0x7fadc4102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 -- 172.21.15.37:0/1134000219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41a8000 con 0x7fadc406bbd0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.730+0000 7fadc9a44700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c9b0 tx=0x7fadc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.731+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0007b00 con 0x7fadc406bbd0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.731+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41a82f0 con 0x7fadc406bbd0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.731+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc0007ca0 con 0x7fadc406bbd0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.731+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0018660 con 0x7fadc406bbd0 2024-04-26T18:21:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.731+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a8880 con 0x7fadc406bbd0 2024-04-26T18:21:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.732+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc406a110 con 0x7fadc406bbd0 2024-04-26T18:21:10.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.734+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fadc000f450 con 0x7fadc406bbd0 2024-04-26T18:21:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.734+0000 7fadbaffd700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 0x7fadb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.734+0000 7fadc9243700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 0x7fadb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.734+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7fadc00d04b0 con 0x7fadc406bbd0 2024-04-26T18:21:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.735+0000 7fadc9243700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 0x7fadb00a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.738+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadc0014030 con 0x7fadc406bbd0 2024-04-26T18:21:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:10.995+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fadc402e490 con 0x7fadc406bbd0 2024-04-26T18:21:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.903+0000 7fadbaffd700 1 -- 172.21.15.37:0/1134000219 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fadc009d0d0 con 0x7fadc406bbd0 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.906+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 msgr2=0x7fadb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.906+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 0x7fadb00a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadb4006fd0 tx=0x7fadb4008040 comp rx=0 tx=0).stop 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.906+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 msgr2=0x7fadc4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.906+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c9b0 tx=0x7fadc000ccc0 comp rx=0 tx=0).stop 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 shutdown_connections 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fadb00a3c40 0x7fadb00a6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fadc41094d0 0x7fadc4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadc406bbd0 0x7fadc4102390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 --2- 172.21.15.37:0/1134000219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fadc406c590 0x7fadc41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 >> 172.21.15.37:0/1134000219 conn(0x7fadc40793c0 msgr2=0x7fadc4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 shutdown_connections 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:11.907+0000 7fadcbca8700 1 -- 172.21.15.37:0/1134000219 wait complete. 2024-04-26T18:21:11.908 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T18:21:12.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:11 smithi037 ceph-mon[29171]: pgmap v294: 97 pgs: 26 creating+peering, 55 active+clean, 16 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:12.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:11 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:21:12.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:11 smithi037 ceph-mon[29171]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:21:12.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:11 smithi120 ceph-mon[32244]: pgmap v294: 97 pgs: 26 creating+peering, 55 active+clean, 16 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:12.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:11 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:21:12.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:11 smithi120 ceph-mon[32244]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:21:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:11 smithi170 ceph-mon[32697]: pgmap v294: 97 pgs: 26 creating+peering, 55 active+clean, 16 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:11 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:21:12.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:11 smithi170 ceph-mon[32697]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:21:12.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:21:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:13.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:12 smithi037 ceph-mon[29171]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:21:13.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:21:13.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:12 smithi037 ceph-mon[29171]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:21:13.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:12 smithi037 ceph-mon[29171]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:21:13.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:12 smithi120 ceph-mon[32244]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:21:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:21:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:12 smithi120 ceph-mon[32244]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:21:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:12 smithi120 ceph-mon[32244]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:21:13.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:12 smithi170 ceph-mon[32697]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:21:13.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1134000219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:21:13.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:12 smithi170 ceph-mon[32697]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:21:13.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:12 smithi170 ceph-mon[32697]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:21:13.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: pgmap v296: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:13.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:21:13.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: Cluster is now healthy 2024-04-26T18:21:13.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:active 2024-04-26T18:21:13.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:13.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:13 smithi037 ceph-mon[29171]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:14.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: pgmap v296: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:14.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:21:14.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: Cluster is now healthy 2024-04-26T18:21:14.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:active 2024-04-26T18:21:14.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:14.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:13 smithi120 ceph-mon[32244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: pgmap v296: 97 pgs: 20 creating+peering, 77 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:21:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:21:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: Cluster is now healthy 2024-04-26T18:21:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] up:active 2024-04-26T18:21:14.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:14.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:13 smithi170 ceph-mon[32697]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 -- 172.21.15.37:0/3813324217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 msgr2=0x7f6ddc1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 --2- 172.21.15.37:0/3813324217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc1102c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400b0a0 tx=0x7f6dd400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 -- 172.21.15.37:0/3813324217 shutdown_connections 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 --2- 172.21.15.37:0/3813324217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc1102c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 --2- 172.21.15.37:0/3813324217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc109480 0x7f6ddc109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 --2- 172.21.15.37:0/3813324217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc1034e0 0x7f6ddc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 -- 172.21.15.37:0/3813324217 >> 172.21.15.37:0/3813324217 conn(0x7f6ddc07a5e0 msgr2=0x7f6ddc07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 -- 172.21.15.37:0/3813324217 shutdown_connections 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.489+0000 7f6de402e700 1 -- 172.21.15.37:0/3813324217 wait complete. 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 Processor -- start 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 -- start start 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc1034e0 0x7f6ddc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc109480 0x7f6ddc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc1135c0 con 0x7f6ddc103ea0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc113440 con 0x7f6ddc1034e0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de402e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc113740 con 0x7f6ddc109480 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37546/0 (socket says 172.21.15.37:37546) 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 -- 172.21.15.37:0/3335351212 learned_addr learned my addr 172.21.15.37:0/3335351212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de1dca700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc1034e0 0x7f6ddc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de25cb700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc109480 0x7f6ddc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 -- 172.21.15.37:0/3335351212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc109480 msgr2=0x7f6ddc1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc109480 0x7f6ddc1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 -- 172.21.15.37:0/3335351212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc1034e0 msgr2=0x7f6ddc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc1034e0 0x7f6ddc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.490+0000 7f6de15c9700 1 -- 172.21.15.37:0/3335351212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc19a040 con 0x7f6ddc103ea0 2024-04-26T18:21:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6de15c9700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00d980 tx=0x7f6dcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc017070 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc19a330 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dcc005020 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00fa10 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.491+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1a65a0 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.492+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc104b30 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.493+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6dcc004a30 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.493+0000 7f6dd2ffd700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 0x7f6dc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.494+0000 7f6de1dca700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 0x7f6dc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.494+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6dcc0d01f0 con 0x7f6ddc103ea0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.494+0000 7f6de1dca700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 0x7f6dc80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc108760 tx=0x7f6dd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.498+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6dcc020020 con 0x7f6ddc103ea0 2024-04-26T18:21:14.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.713+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ddc02e490 con 0x7f6ddc103ea0 2024-04-26T18:21:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.715+0000 7f6dd2ffd700 1 -- 172.21.15.37:0/3335351212 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22179 (secure 0 0 0) 0x7f6dcc09cc80 con 0x7f6ddc103ea0 2024-04-26T18:21:14.715 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:14.716 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:21:10.996894+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:21:14.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 msgr2=0x7f6dc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:14.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 0x7f6dc80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc108760 tx=0x7f6dd800b410 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 msgr2=0x7f6ddc19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00d980 tx=0x7f6dcc00dc90 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 shutdown_connections 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6dc80a3c40 0x7f6dc80a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ddc109480 0x7f6ddc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ddc1034e0 0x7f6ddc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 --2- 172.21.15.37:0/3335351212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc103ea0 0x7f6ddc19fc10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 >> 172.21.15.37:0/3335351212 conn(0x7f6ddc07a5e0 msgr2=0x7f6ddc1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 shutdown_connections 2024-04-26T18:21:14.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:14.717+0000 7f6de402e700 1 -- 172.21.15.37:0/3335351212 wait complete. 2024-04-26T18:21:15.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:14 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3335351212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:15.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:14 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3335351212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:15.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:14 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3335351212' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:15.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:21:15.644 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:16.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:15 smithi037 ceph-mon[29171]: pgmap v297: 97 pgs: 97 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2024-04-26T18:21:16.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:15 smithi120 ceph-mon[32244]: pgmap v297: 97 pgs: 97 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2024-04-26T18:21:16.253 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:21:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: 2024-04-26T18:21:15.958+0000 7f13c960d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T18:21:16.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:15 smithi170 ceph-mon[32697]: pgmap v297: 97 pgs: 97 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 -- 172.21.15.37:0/1611913569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102310 msgr2=0x7fb350102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 --2- 172.21.15.37:0/1611913569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102310 0x7fb350102790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb344009990 tx=0x7fb344009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 -- 172.21.15.37:0/1611913569 shutdown_connections 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 --2- 172.21.15.37:0/1611913569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350102cd0 0x7fb35010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 --2- 172.21.15.37:0/1611913569 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350108380 0x7fb350108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 --2- 172.21.15.37:0/1611913569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102310 0x7fb350102790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 -- 172.21.15.37:0/1611913569 >> 172.21.15.37:0/1611913569 conn(0x7fb350079290 msgr2=0x7fb3500796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 -- 172.21.15.37:0/1611913569 shutdown_connections 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.236+0000 7fb3564c9700 1 -- 172.21.15.37:0/1611913569 wait complete. 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 Processor -- start 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 -- start start 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 0x7fb35019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 0x7fb350199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501124d0 con 0x7fb350102cd0 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb350112350 con 0x7fb350102310 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb3564c9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb350112650 con 0x7fb350108380 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58460/0 (socket says 172.21.15.37:58460) 2024-04-26T18:21:17.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 -- 172.21.15.37:0/2455819950 learned_addr learned my addr 172.21.15.37:0/2455819950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34ffff700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 0x7fb35019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb354a66700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 0x7fb350199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 -- 172.21.15.37:0/2455819950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 msgr2=0x7fb350199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 0x7fb350199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 -- 172.21.15.37:0/2455819950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 msgr2=0x7fb35019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 0x7fb35019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34f7fe700 1 -- 172.21.15.37:0/2455819950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35019a070 con 0x7fb350102cd0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb354a66700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 0x7fb350199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.237+0000 7fb34ffff700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 0x7fb35019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb34f7fe700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb344000c00 tx=0x7fb3440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344027070 con 0x7fb350102cd0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35019a300 con 0x7fb350102cd0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3440041d0 con 0x7fb350102cd0 2024-04-26T18:21:17.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34400f460 con 0x7fb350102cd0 2024-04-26T18:21:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.238+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3501a6620 con 0x7fb350102cd0 2024-04-26T18:21:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.239+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb350103a30 con 0x7fb350102cd0 2024-04-26T18:21:17.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.240+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb34402a070 con 0x7fb350102cd0 2024-04-26T18:21:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.241+0000 7fb34d7fa700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 0x7fb3380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.241+0000 7fb34ffff700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 0x7fb3380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.241+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb3440d1330 con 0x7fb350102cd0 2024-04-26T18:21:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.241+0000 7fb34ffff700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 0x7fb3380a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb340005e20 tx=0x7fb340005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.245+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb34405eff0 con 0x7fb350102cd0 2024-04-26T18:21:17.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.458+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3501a67f0 con 0x7fb350102cd0 2024-04-26T18:21:17.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.459+0000 7fb34d7fa700 1 -- 172.21.15.37:0/2455819950 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22179 (secure 0 0 0) 0x7fb344013070 con 0x7fb350102cd0 2024-04-26T18:21:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:17.460 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:21:10.996894+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 msgr2=0x7fb3380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 0x7fb3380a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb340005e20 tx=0x7fb340005db0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 msgr2=0x7fb35019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb344000c00 tx=0x7fb3440047b0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 shutdown_connections 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb3380a3c40 0x7fb3380a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb350108380 0x7fb350199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb350102310 0x7fb35019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 --2- 172.21.15.37:0/2455819950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb350102cd0 0x7fb35019fc90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 >> 172.21.15.37:0/2455819950 conn(0x7fb350079290 msgr2=0x7fb3501015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 shutdown_connections 2024-04-26T18:21:17.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:17.462+0000 7fb3564c9700 1 -- 172.21.15.37:0/2455819950 wait complete. 2024-04-26T18:21:18.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:17 smithi037 ceph-mon[29171]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T18:21:18.136 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:17 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2455819950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:18.173 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T18:21:18.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:17 smithi120 ceph-mon[32244]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T18:21:18.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:17 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2455819950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:18.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:17 smithi170 ceph-mon[32697]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T18:21:18.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:17 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2455819950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:18.429 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:20.188 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:19 smithi037 ceph-mon[29171]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T18:21:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:19 smithi120 ceph-mon[32244]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 -- 172.21.15.37:0/2968416111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103e30 msgr2=0x7ff554110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 --2- 172.21.15.37:0/2968416111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103e30 0x7ff554110290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00b0a0 tx=0x7ff54c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 -- 172.21.15.37:0/2968416111 shutdown_connections 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 --2- 172.21.15.37:0/2968416111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103e30 0x7ff554110290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 --2- 172.21.15.37:0/2968416111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 0x7ff5541098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 --2- 172.21.15.37:0/2968416111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103470 0x7ff5541038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 -- 172.21.15.37:0/2968416111 >> 172.21.15.37:0/2968416111 conn(0x7ff55407a580 msgr2=0x7ff55407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 -- 172.21.15.37:0/2968416111 shutdown_connections 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.298+0000 7ff5599e4700 1 -- 172.21.15.37:0/2968416111 wait complete. 2024-04-26T18:21:20.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 Processor -- start 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 -- start start 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 0x7ff55419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 0x7ff554199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541135c0 con 0x7ff554103470 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff554113440 con 0x7ff554103e30 2024-04-26T18:21:20.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff5599e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff554113740 con 0x7ff5541094c0 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff552ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58474/0 (socket says 172.21.15.37:58474) 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.299+0000 7ff552ffd700 1 -- 172.21.15.37:0/3493961658 learned_addr learned my addr 172.21.15.37:0/3493961658 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5527fc700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 0x7ff55419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5537fe700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 0x7ff554199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 -- 172.21.15.37:0/3493961658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 msgr2=0x7ff554199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 0x7ff554199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 -- 172.21.15.37:0/3493961658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 msgr2=0x7ff55419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 0x7ff55419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 -- 172.21.15.37:0/3493961658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff554199fe0 con 0x7ff554103470 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5527fc700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 0x7ff55419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff552ffd700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c9b0 tx=0x7ff54400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544007b00 con 0x7ff554103470 2024-04-26T18:21:20.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff544007ca0 con 0x7ff554103470 2024-04-26T18:21:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55419a2d0 con 0x7ff554103470 2024-04-26T18:21:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544018730 con 0x7ff554103470 2024-04-26T18:21:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.300+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff554075650 con 0x7ff554103470 2024-04-26T18:21:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.301+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff554104b60 con 0x7ff554103470 2024-04-26T18:21:20.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.303+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff54400f450 con 0x7ff554103470 2024-04-26T18:21:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.304+0000 7ff5589e2700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 0x7ff53c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.304+0000 7ff5527fc700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 0x7ff53c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.304+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7ff5440d0670 con 0x7ff554103470 2024-04-26T18:21:20.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.304+0000 7ff5527fc700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 0x7ff53c0a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff55407ab80 tx=0x7ff54800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:20.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.308+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff544014030 con 0x7ff554103470 2024-04-26T18:21:20.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:19 smithi170 ceph-mon[32697]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T18:21:20.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.564+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff55402e490 con 0x7ff554103470 2024-04-26T18:21:20.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.920+0000 7ff5589e2700 1 -- 172.21.15.37:0/3493961658 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ff54409d190 con 0x7ff554103470 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 msgr2=0x7ff53c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 0x7ff53c0a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff55407ab80 tx=0x7ff54800b5f0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 msgr2=0x7ff55419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c9b0 tx=0x7ff54400cd70 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 shutdown_connections 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff53c0a3c40 0x7ff53c0a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5541094c0 0x7ff554199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff554103470 0x7ff55419f680 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 --2- 172.21.15.37:0/3493961658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff554103e30 0x7ff55419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.922+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 >> 172.21.15.37:0/3493961658 conn(0x7ff55407a580 msgr2=0x7ff5541020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.923+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 shutdown_connections 2024-04-26T18:21:20.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:20.923+0000 7ff5599e4700 1 -- 172.21.15.37:0/3493961658 wait complete. 2024-04-26T18:21:21.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:20 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:21:21.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:20 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:21:21.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:20 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:21:21.661 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-26T18:21:21.662 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-26T18:21:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:21 smithi037 ceph-mon[29171]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:21 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:21:22.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:21 smithi037 ceph-mon[29171]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:22.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:21 smithi120 ceph-mon[32244]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:22.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:21 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:21:22.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:21 smithi120 ceph-mon[32244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:21 smithi170 ceph-mon[32697]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:21 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3493961658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:21:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:21 smithi170 ceph-mon[32697]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 -- 172.21.15.37:0/3605748079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc714102f50 msgr2=0x7fc714106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 --2- 172.21.15.37:0/3605748079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc714102f50 0x7fc714106fa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc7000099c0 tx=0x7fc700009cd0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 -- 172.21.15.37:0/3605748079 shutdown_connections 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 --2- 172.21.15.37:0/3605748079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 0x7fc71410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 --2- 172.21.15.37:0/3605748079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc7141025a0 0x7fc714102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 --2- 172.21.15.37:0/3605748079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc714102f50 0x7fc714106fa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.611+0000 7fc71ac63700 1 -- 172.21.15.37:0/3605748079 >> 172.21.15.37:0/3605748079 conn(0x7fc7140793c0 msgr2=0x7fc7140797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- 172.21.15.37:0/3605748079 shutdown_connections 2024-04-26T18:21:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- 172.21.15.37:0/3605748079 wait complete. 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 Processor -- start 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- start start 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc714102f50 0x7fc714104580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 0x7fc714104ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc714112510 con 0x7fc7141025a0 2024-04-26T18:21:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc714112390 con 0x7fc714102f50 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.612+0000 7fc71ac63700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc714112690 con 0x7fc714107680 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58494/0 (socket says 172.21.15.37:58494) 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 -- 172.21.15.37:0/2413117585 learned_addr learned my addr 172.21.15.37:0/2413117585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc719200700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 0x7fc714104ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc713fff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc714102f50 0x7fc714104580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 -- 172.21.15.37:0/2413117585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 msgr2=0x7fc714104ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 0x7fc714104ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 -- 172.21.15.37:0/2413117585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc714102f50 msgr2=0x7fc714104580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc714102f50 0x7fc714104580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 -- 172.21.15.37:0/2413117585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141a8080 con 0x7fc7141025a0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc7189ff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d950 tx=0x7fc70800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708016070 con 0x7fc7141025a0 2024-04-26T18:21:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70800fd40 con 0x7fc7141025a0 2024-04-26T18:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7141a8370 con 0x7fc7141025a0 2024-04-26T18:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.613+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70800e3b0 con 0x7fc7141025a0 2024-04-26T18:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.614+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141a8880 con 0x7fc7141025a0 2024-04-26T18:21:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.615+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71406a110 con 0x7fc7141025a0 2024-04-26T18:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.615+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc708019070 con 0x7fc7141025a0 2024-04-26T18:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.616+0000 7fc711ffb700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 0x7fc7040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.616+0000 7fc713fff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 0x7fc7040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.616+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc7080cfd60 con 0x7fc7141025a0 2024-04-26T18:21:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.617+0000 7fc713fff700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 0x7fc7040a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc700006010 tx=0x7fc700005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.621+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc70809c890 con 0x7fc7141025a0 2024-04-26T18:21:23.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.883+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fc7141a8ab0 con 0x7fc7141025a0 2024-04-26T18:21:23.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.924+0000 7fc711ffb700 1 -- 172.21.15.37:0/2413117585 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fc70809c190 con 0x7fc7141025a0 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.926+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 msgr2=0x7fc7040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.926+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 0x7fc7040a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc700006010 tx=0x7fc700005c60 comp rx=0 tx=0).stop 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.926+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 msgr2=0x7fc714104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.926+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d950 tx=0x7fc70800dc60 comp rx=0 tx=0).stop 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 shutdown_connections 2024-04-26T18:21:23.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc7040a3c40 0x7fc7040a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc714107680 0x7fc714104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7141025a0 0x7fc714104040 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 --2- 172.21.15.37:0/2413117585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc714102f50 0x7fc714104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 >> 172.21.15.37:0/2413117585 conn(0x7fc7140793c0 msgr2=0x7fc714108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 shutdown_connections 2024-04-26T18:21:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:23.927+0000 7fc71ac63700 1 -- 172.21.15.37:0/2413117585 wait complete. 2024-04-26T18:21:24.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:23 smithi037 ceph-mon[29171]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:21:24.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:23 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:21:24.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:23 smithi120 ceph-mon[32244]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:21:24.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:23 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:21:24.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:23 smithi170 ceph-mon[32697]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:21:24.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:23 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:21:24.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T18:21:24.917 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:25.191 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:21:25.191 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:21:25.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:21:25.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:25.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T18:21:25.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:24 smithi037 ceph-mon[29171]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:25.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:21:25.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:21:25.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:21:25.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:25.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T18:21:25.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:24 smithi120 ceph-mon[32244]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:25.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:21:25.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2413117585' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:21:25.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:21:25.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:21:25.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T18:21:25.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:24 smithi170 ceph-mon[32697]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: pgmap v302: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:active 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: Cluster is now healthy 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T18:21:26.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:25 smithi037 ceph-mon[29171]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T18:21:26.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: pgmap v302: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:active 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: Cluster is now healthy 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-mon[32244]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: 2024-04-26T18:21:25.958+0000 7f13c960d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:21:26.253 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:21:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: 2024-04-26T18:21:25.958+0000 7f13c960d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:21:26.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: pgmap v302: 97 pgs: 97 active+clean; 580 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:21:26.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] up:active 2024-04-26T18:21:26.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:21:26.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:21:26.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: Cluster is now healthy 2024-04-26T18:21:26.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T18:21:26.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T18:21:26.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:25 smithi170 ceph-mon[32697]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T18:21:26.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 -- 172.21.15.37:0/4106078932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 msgr2=0x7fc948110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 --2- 172.21.15.37:0/4106078932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc948110290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b0a0 tx=0x7fc94000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 -- 172.21.15.37:0/4106078932 shutdown_connections 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 --2- 172.21.15.37:0/4106078932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc948110290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 --2- 172.21.15.37:0/4106078932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9481094c0 0x7fc9481098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 --2- 172.21.15.37:0/4106078932 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc948103470 0x7fc9481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 -- 172.21.15.37:0/4106078932 >> 172.21.15.37:0/4106078932 conn(0x7fc94807a580 msgr2=0x7fc94807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 -- 172.21.15.37:0/4106078932 shutdown_connections 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.548+0000 7fc94e15a700 1 -- 172.21.15.37:0/4106078932 wait complete. 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 Processor -- start 2024-04-26T18:21:26.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 -- start start 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc948103470 0x7fc9481a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 0x7fc94819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc948113610 con 0x7fc948103e30 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc948113490 con 0x7fc9481094c0 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.549+0000 7fc94e15a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc948113790 con 0x7fc948103470 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46174/0 (socket says 172.21.15.37:46174) 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 -- 172.21.15.37:0/2898408516 learned_addr learned my addr 172.21.15.37:0/2898408516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc947fff700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 0x7fc94819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc9477fe700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc948103470 0x7fc9481a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 -- 172.21.15.37:0/2898408516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc948103470 msgr2=0x7fc9481a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc948103470 0x7fc9481a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 -- 172.21.15.37:0/2898408516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 msgr2=0x7fc94819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 0x7fc94819bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 -- 172.21.15.37:0/2898408516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94819c3f0 con 0x7fc948103e30 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc947fff700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 0x7fc94819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc946ffd700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00eb30 tx=0x7fc93c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00cbd0 con 0x7fc948103e30 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93c0044d0 con 0x7fc948103e30 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94819c6e0 con 0x7fc948103e30 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c003e30 con 0x7fc948103e30 2024-04-26T18:21:26.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.550+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481a89c0 con 0x7fc948103e30 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.551+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc948104b60 con 0x7fc948103e30 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.552+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc93c004000 con 0x7fc948103e30 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.553+0000 7fc944ff9700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.553+0000 7fc9477fe700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.553+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc93c0d0600 con 0x7fc948103e30 2024-04-26T18:21:26.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.554+0000 7fc9477fe700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 0x7fc9300a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc938005950 tx=0x7fc93800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:26.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.558+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc93c0042f0 con 0x7fc948103e30 2024-04-26T18:21:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.814+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc9481a8b90 con 0x7fc948103e30 2024-04-26T18:21:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.945+0000 7fc944ff9700 1 -- 172.21.15.37:0/2898408516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fc93c09d130 con 0x7fc948103e30 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 msgr2=0x7fc9300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 0x7fc9300a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc938005950 tx=0x7fc93800a300 comp rx=0 tx=0).stop 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 msgr2=0x7fc9481a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00eb30 tx=0x7fc93c00ee40 comp rx=0 tx=0).stop 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 shutdown_connections 2024-04-26T18:21:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9300a3c40 0x7fc9300a6100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9481094c0 0x7fc94819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc948103470 0x7fc9481a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 --2- 172.21.15.37:0/2898408516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc948103e30 0x7fc9481a1f80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 >> 172.21.15.37:0/2898408516 conn(0x7fc94807a580 msgr2=0x7fc948101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 shutdown_connections 2024-04-26T18:21:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:26.948+0000 7fc94e15a700 1 -- 172.21.15.37:0/2898408516 wait complete. 2024-04-26T18:21:27.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:26 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:active 2024-04-26T18:21:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:26 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:26 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:21:27.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:26 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:active 2024-04-26T18:21:27.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:26 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:27.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:26 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:21:27.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:26 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] up:active 2024-04-26T18:21:27.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:26 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:27.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:26 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:21:27.609 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T18:21:27.858 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:28.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:27 smithi037 ceph-mon[29171]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T18:21:28.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:27 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:21:28.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:27 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:28.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:27 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:28.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:27 smithi120 ceph-mon[32244]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T18:21:28.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:27 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:21:28.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:27 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:28.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:27 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:28.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:27 smithi170 ceph-mon[32697]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T18:21:28.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:27 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2898408516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:21:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:27 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T18:21:28.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:27 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- 172.21.15.37:0/2317923746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c0107640 msgr2=0x7fd9c010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/2317923746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c0107640 0x7fd9c010f210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b0d0 tx=0x7fd9b800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- 172.21.15.37:0/2317923746 shutdown_connections 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/2317923746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c0107640 0x7fd9c010f210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/2317923746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c01025b0 0x7fd9c0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/2317923746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 0x7fd9c0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- 172.21.15.37:0/2317923746 >> 172.21.15.37:0/2317923746 conn(0x7fd9c00792d0 msgr2=0x7fd9c00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- 172.21.15.37:0/2317923746 shutdown_connections 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- 172.21.15.37:0/2317923746 wait complete. 2024-04-26T18:21:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 Processor -- start 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.408+0000 7fd9c5b95700 1 -- start start 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 0x7fd9c0104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c0107640 0x7fd9c0104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c01124e0 con 0x7fd9c01025b0 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0112360 con 0x7fd9c0102f60 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9c5b95700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0112660 con 0x7fd9c0107640 2024-04-26T18:21:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46192/0 (socket says 172.21.15.37:46192) 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 -- 172.21.15.37:0/3025026471 learned_addr learned my addr 172.21.15.37:0/3025026471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9beffd700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 0x7fd9c0104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bffff700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c0107640 0x7fd9c0104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 -- 172.21.15.37:0/3025026471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c0107640 msgr2=0x7fd9c0104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c0107640 0x7fd9c0104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 -- 172.21.15.37:0/3025026471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 msgr2=0x7fd9c0104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 0x7fd9c0104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 -- 172.21.15.37:0/3025026471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c01a8000 con 0x7fd9c01025b0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bf7fe700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000d980 tx=0x7fd9b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.409+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0016070 con 0x7fd9c01025b0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.410+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c01a82f0 con 0x7fd9c01025b0 2024-04-26T18:21:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.410+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b0004830 con 0x7fd9c01025b0 2024-04-26T18:21:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.410+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0005540 con 0x7fd9c01025b0 2024-04-26T18:21:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.410+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c01a8880 con 0x7fd9c01025b0 2024-04-26T18:21:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.410+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c006a110 con 0x7fd9c01025b0 2024-04-26T18:21:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.412+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9b0004220 con 0x7fd9c01025b0 2024-04-26T18:21:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.412+0000 7fd9bcff9700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 0x7fd9a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.412+0000 7fd9beffd700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 0x7fd9a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.412+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd9b00d0570 con 0x7fd9c01025b0 2024-04-26T18:21:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.413+0000 7fd9beffd700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 0x7fd9a80a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0108c30 tx=0x7fd9b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.416+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9b001f020 con 0x7fd9c01025b0 2024-04-26T18:21:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.672+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd9c002e490 con 0x7fd9c01025b0 2024-04-26T18:21:30.353 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:29 smithi170 ceph-mon[32697]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-26T18:21:30.354 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:29 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:21:30.354 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:29 smithi120 ceph-mon[32244]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-26T18:21:30.354 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:29 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:21:30.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.949+0000 7fd9bcff9700 1 -- 172.21.15.37:0/3025026471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fd9b009d0a0 con 0x7fd9c01025b0 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 msgr2=0x7fd9a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 0x7fd9a80a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0108c30 tx=0x7fd9b400b410 comp rx=0 tx=0).stop 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 msgr2=0x7fd9c0104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000d980 tx=0x7fd9b000dc90 comp rx=0 tx=0).stop 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 shutdown_connections 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fd9a80a3c40 0x7fd9a80a6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9c0107640 0x7fd9c0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c01025b0 0x7fd9c0104000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 --2- 172.21.15.37:0/3025026471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9c0102f60 0x7fd9c0104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 >> 172.21.15.37:0/3025026471 conn(0x7fd9c00792d0 msgr2=0x7fd9c006be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 shutdown_connections 2024-04-26T18:21:30.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:29.953+0000 7fd9c5b95700 1 -- 172.21.15.37:0/3025026471 wait complete. 2024-04-26T18:21:30.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:29 smithi037 ceph-mon[29171]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-26T18:21:30.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:29 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:21:30.746 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-26T18:21:30.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T18:21:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:30 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:21:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:30 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:31.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:30 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:21:31.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:30 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:31.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:30 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3025026471' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:21:31.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:30 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:32.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:31 smithi037 ceph-mon[29171]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:21:32.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:31 smithi120 ceph-mon[32244]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:21:32.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:31 smithi170 ceph-mon[32697]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3612797281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c406bad0 msgr2=0x7f52c41102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3612797281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c406bad0 0x7f52c41102e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009990 tx=0x7f52bc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3612797281 shutdown_connections 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3612797281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c406bad0 0x7f52c41102e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3612797281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c41094a0 0x7f52c4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3612797281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 0x7f52c406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3612797281 >> 172.21.15.37:0/3612797281 conn(0x7f52c40792d0 msgr2=0x7f52c40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3612797281 shutdown_connections 2024-04-26T18:21:32.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.596+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3612797281 wait complete. 2024-04-26T18:21:32.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 Processor -- start 2024-04-26T18:21:32.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 -- start start 2024-04-26T18:21:32.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 0x7f52c4103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 0x7f52c41023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41135f0 con 0x7f52c41094a0 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4113470 con 0x7f52c406b130 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c8ae0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4113770 con 0x7f52c406bad0 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46214/0 (socket says 172.21.15.37:46214) 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.597+0000 7f52c1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 0x7f52c41023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 -- 172.21.15.37:0/3152680220 learned_addr learned my addr 172.21.15.37:0/3152680220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c259c700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 0x7f52c4103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 -- 172.21.15.37:0/3152680220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 msgr2=0x7f52c41023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 0x7f52c41023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 -- 172.21.15.37:0/3152680220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 msgr2=0x7f52c4103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 0x7f52c4103fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 -- 172.21.15.37:0/3152680220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c41ac450 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c1d9b700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 0x7f52c41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c2d9d700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0047b0 tx=0x7f52bc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004830 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc005020 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.598+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c41ac680 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.599+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc003cd0 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.599+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c41aca70 con 0x7f52c41094a0 2024-04-26T18:21:32.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.600+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c4104b50 con 0x7f52c41094a0 2024-04-26T18:21:32.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.601+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52bc005490 con 0x7f52c41094a0 2024-04-26T18:21:32.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.601+0000 7f52b37fe700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:32.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.602+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f52bc0d10f0 con 0x7f52c41094a0 2024-04-26T18:21:32.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.602+0000 7f52c259c700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:32.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.602+0000 7f52c259c700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 0x7f52ac0a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009550 tx=0x7f52b40091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:32.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.605+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52bc013070 con 0x7f52c41094a0 2024-04-26T18:21:32.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.848+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f52c4063860 con 0x7f52ac0a3c40 2024-04-26T18:21:32.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.874+0000 7f52b37fe700 1 -- 172.21.15.37:0/3152680220 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52c4063860 con 0x7f52ac0a3c40 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 msgr2=0x7f52ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 0x7f52ac0a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009550 tx=0x7f52b40091b0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 msgr2=0x7f52c4102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0047b0 tx=0x7f52bc005ef0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 shutdown_connections 2024-04-26T18:21:32.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52c41094a0 0x7f52c4102a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c406b130 0x7f52c4103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 --2- 172.21.15.37:0/3152680220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52c406bad0 0x7f52c41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 >> 172.21.15.37:0/3152680220 conn(0x7f52c40792d0 msgr2=0x7f52c4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 shutdown_connections 2024-04-26T18:21:32.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:32.876+0000 7f52c8ae0700 1 -- 172.21.15.37:0/3152680220 wait complete. 2024-04-26T18:21:33.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:21:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:21:32.853+0000 7fc162e3a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:21:33.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:21:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:21:32.853+0000 7fc162e3a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:21:33.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:21:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:21:32.853+0000 7fc162e3a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:21:33.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:21:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:21:32.853+0000 7fc162e3a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:21:33.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:21:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:21:32.853+0000 7fc162e3a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:21:33.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T18:21:33.918 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:34.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:33 smithi037 ceph-mon[29171]: pgmap v306: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T18:21:34.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:33 smithi037 ceph-mon[29171]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:34.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:33 smithi120 ceph-mon[32244]: pgmap v306: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T18:21:34.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:33 smithi120 ceph-mon[32244]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:34.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:33 smithi170 ceph-mon[32697]: pgmap v306: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-26T18:21:34.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:33 smithi170 ceph-mon[32697]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:35.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 -- 172.21.15.37:0/733385646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 msgr2=0x7fc98806b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:35.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 --2- 172.21.15.37:0/733385646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98806b590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009990 tx=0x7fc97c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 -- 172.21.15.37:0/733385646 shutdown_connections 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 --2- 172.21.15.37:0/733385646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 0x7fc9881102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 --2- 172.21.15.37:0/733385646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 0x7fc988109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 --2- 172.21.15.37:0/733385646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98806b590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 -- 172.21.15.37:0/733385646 >> 172.21.15.37:0/733385646 conn(0x7fc9880792d0 msgr2=0x7fc9880796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 -- 172.21.15.37:0/733385646 shutdown_connections 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.814+0000 7fc98f139700 1 -- 172.21.15.37:0/733385646 wait complete. 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 Processor -- start 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 -- start start 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 0x7fc98819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:35.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 0x7fc988199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc988113690 con 0x7fc98806b130 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc988113510 con 0x7fc98806bad0 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98f139700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc988113810 con 0x7fc9881094a0 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98ced5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98ced5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39388/0 (socket says 172.21.15.37:39388) 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.815+0000 7fc98ced5700 1 -- 172.21.15.37:0/3780791156 learned_addr learned my addr 172.21.15.37:0/3780791156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98d6d6700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 0x7fc988199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 -- 172.21.15.37:0/3780791156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 msgr2=0x7fc988199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc987fff700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 0x7fc98819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 0x7fc988199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 -- 172.21.15.37:0/3780791156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 msgr2=0x7fc98819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:35.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 0x7fc98819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 -- 172.21.15.37:0/3780791156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98819a1c0 con 0x7fc98806b130 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98ced5700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b530 tx=0x7fc97800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978015020 con 0x7fc98806b130 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978011ce0 con 0x7fc98806b130 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98819a4b0 con 0x7fc98806b130 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978010d20 con 0x7fc98806b130 2024-04-26T18:21:35.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.816+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881a6770 con 0x7fc98806b130 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.817+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98810ba10 con 0x7fc98806b130 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.818+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc978018020 con 0x7fc98806b130 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.819+0000 7fc985ffb700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 0x7fc9700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.819+0000 7fc987fff700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 0x7fc9700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.819+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fc9780cfb60 con 0x7fc98806b130 2024-04-26T18:21:35.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.819+0000 7fc987fff700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 0x7fc9700a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005cd0 tx=0x7fc97c005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:35.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:35.824+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc97801f020 con 0x7fc98806b130 2024-04-26T18:21:36.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.068+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc988063860 con 0x7fc9700a3cd0 2024-04-26T18:21:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.095+0000 7fc985ffb700 1 -- 172.21.15.37:0/3780791156 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc988063860 con 0x7fc9700a3cd0 2024-04-26T18:21:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 msgr2=0x7fc9700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 0x7fc9700a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc97c005cd0 tx=0x7fc97c005c40 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 msgr2=0x7fc98819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b530 tx=0x7fc97800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 shutdown_connections 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc9700a3cd0 0x7fc9700a6190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9881094a0 0x7fc988199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc98806b130 0x7fc98819f8a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 --2- 172.21.15.37:0/3780791156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc98806bad0 0x7fc98819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 >> 172.21.15.37:0/3780791156 conn(0x7fc9880792d0 msgr2=0x7fc9881004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 shutdown_connections 2024-04-26T18:21:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:36.097+0000 7fc98f139700 1 -- 172.21.15.37:0/3780791156 wait complete. 2024-04-26T18:21:36.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:35 smithi037 ceph-mon[29171]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:36.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:35 smithi037 ceph-mon[29171]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-26T18:21:36.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:35 smithi120 ceph-mon[32244]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:36.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:35 smithi120 ceph-mon[32244]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-26T18:21:36.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:35 smithi170 ceph-mon[32697]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:36.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:35 smithi170 ceph-mon[32697]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-26T18:21:36.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T18:21:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:37.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:36 smithi037 ceph-mon[29171]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:37.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:36 smithi120 ceph-mon[32244]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:37.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:36 smithi170 ceph-mon[32697]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:38.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:37 smithi037 ceph-mon[29171]: pgmap v308: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-26T18:21:38.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:37 smithi120 ceph-mon[32244]: pgmap v308: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-26T18:21:38.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:37 smithi170 ceph-mon[32697]: pgmap v308: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-04-26T18:21:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 -- 172.21.15.37:0/2467861140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806c360 msgr2=0x7fb0e81102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/2467861140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806c360 0x7fb0e81102d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000b0a0 tx=0x7fb0e000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 -- 172.21.15.37:0/2467861140 shutdown_connections 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/2467861140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806c360 0x7fb0e81102d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/2467861140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e8109490 0x7fb0e8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/2467861140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e806b9c0 0x7fb0e806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 -- 172.21.15.37:0/2467861140 >> 172.21.15.37:0/2467861140 conn(0x7fb0e8079290 msgr2=0x7fb0e80796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 -- 172.21.15.37:0/2467861140 shutdown_connections 2024-04-26T18:21:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.810+0000 7fb0eff0c700 1 -- 172.21.15.37:0/2467861140 wait complete. 2024-04-26T18:21:38.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 Processor -- start 2024-04-26T18:21:38.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 -- start start 2024-04-26T18:21:38.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e806c360 0x7fb0e819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e8109490 0x7fb0e81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8113610 con 0x7fb0e806b9c0 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8113490 con 0x7fb0e8109490 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0eff0c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8113790 con 0x7fb0e806c360 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0edca8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0edca8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39400/0 (socket says 172.21.15.37:39400) 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0edca8700 1 -- 172.21.15.37:0/1192897183 learned_addr learned my addr 172.21.15.37:0/1192897183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0ee4a9700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e8109490 0x7fb0e81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0ed4a7700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e806c360 0x7fb0e819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.811+0000 7fb0edca8700 1 -- 172.21.15.37:0/1192897183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e806c360 msgr2=0x7fb0e819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0edca8700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e806c360 0x7fb0e819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0edca8700 1 -- 172.21.15.37:0/1192897183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e8109490 msgr2=0x7fb0e81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0edca8700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e8109490 0x7fb0e81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0edca8700 1 -- 172.21.15.37:0/1192897183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e81a0d50 con 0x7fb0e806b9c0 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0edca8700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b530 tx=0x7fb0e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4015020 con 0x7fb0e806b9c0 2024-04-26T18:21:38.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e4011ce0 con 0x7fb0e806b9c0 2024-04-26T18:21:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e81a1040 con 0x7fb0e806b9c0 2024-04-26T18:21:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4010e40 con 0x7fb0e806b9c0 2024-04-26T18:21:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.812+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a88c0 con 0x7fb0e806b9c0 2024-04-26T18:21:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.813+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e806a110 con 0x7fb0e806b9c0 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.814+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0e4018020 con 0x7fb0e806b9c0 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.815+0000 7fb0deffd700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 0x7fb0d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.815+0000 7fb0ed4a7700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 0x7fb0d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.815+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb0e40cfc30 con 0x7fb0e806b9c0 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.815+0000 7fb0ed4a7700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 0x7fb0d40a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005950 tx=0x7fb0d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:38.819+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0e401f020 con 0x7fb0e806b9c0 2024-04-26T18:21:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.062+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb0e819d0a0 con 0x7fb0d40a3cd0 2024-04-26T18:21:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.068+0000 7fb0deffd700 1 -- 172.21.15.37:0/1192897183 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb0e819d0a0 con 0x7fb0d40a3cd0 2024-04-26T18:21:39.068 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/21b3687a-a390-4360-b18e-ea6b2e02d514 2024-04-26T18:21:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 msgr2=0x7fb0d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 0x7fb0d40a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005950 tx=0x7fb0d80058e0 comp rx=0 tx=0).stop 2024-04-26T18:21:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 msgr2=0x7fb0e819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b530 tx=0x7fb0e400b840 comp rx=0 tx=0).stop 2024-04-26T18:21:39.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 shutdown_connections 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0d40a3cd0 0x7fb0d40a6190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0e8109490 0x7fb0e81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0e806b9c0 0x7fb0e819bd40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 --2- 172.21.15.37:0/1192897183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e806c360 0x7fb0e819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 >> 172.21.15.37:0/1192897183 conn(0x7fb0e8079290 msgr2=0x7fb0e810e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 shutdown_connections 2024-04-26T18:21:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:39.070+0000 7fb0eff0c700 1 -- 172.21.15.37:0/1192897183 wait complete. 2024-04-26T18:21:39.991 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T18:21:40.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:39 smithi037 ceph-mon[29171]: pgmap v309: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:21:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:39 smithi037 ceph-mon[29171]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:40.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:39 smithi120 ceph-mon[32244]: pgmap v309: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:21:40.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:39 smithi120 ceph-mon[32244]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:39 smithi170 ceph-mon[32697]: pgmap v309: 97 pgs: 97 active+clean; 588 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:21:40.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:39 smithi170 ceph-mon[32697]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 -- 172.21.15.37:0/922297304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b81094d0 msgr2=0x7f17b81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/922297304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b81094d0 0x7f17b81098b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f17a80099c0 tx=0x7f17a8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 -- 172.21.15.37:0/922297304 shutdown_connections 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/922297304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 0x7f17b81102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/922297304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b81094d0 0x7f17b81098b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/922297304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b8101970 0x7f17b8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 -- 172.21.15.37:0/922297304 >> 172.21.15.37:0/922297304 conn(0x7f17b8078920 msgr2=0x7f17b80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 -- 172.21.15.37:0/922297304 shutdown_connections 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 -- 172.21.15.37:0/922297304 wait complete. 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.058+0000 7f17bd9e4700 1 Processor -- start 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 -- start start 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 0x7f17b8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 0x7f17b81035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17b8113660 con 0x7f17b8101970 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81134e0 con 0x7f17b81094d0 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17bd9e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f17b81137e0 con 0x7f17b8102330 2024-04-26T18:21:42.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39418/0 (socket says 172.21.15.37:39418) 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 -- 172.21.15.37:0/2227627453 learned_addr learned my addr 172.21.15.37:0/2227627453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b77fe700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 0x7f17b81035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b67fc700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 0x7f17b8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 -- 172.21.15.37:0/2227627453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 msgr2=0x7f17b8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 0x7f17b8103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 -- 172.21.15.37:0/2227627453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 msgr2=0x7f17b81035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 0x7f17b81035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 -- 172.21.15.37:0/2227627453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b81a8080 con 0x7f17b8101970 2024-04-26T18:21:42.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b77fe700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 0x7f17b81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.059+0000 7f17b6ffd700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b580 tx=0x7f17a80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.060+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8027070 con 0x7f17b8101970 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.060+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b81a8310 con 0x7f17b8101970 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.060+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a8004020 con 0x7f17b8101970 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.060+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8022e30 con 0x7f17b8101970 2024-04-26T18:21:42.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.060+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81a87c0 con 0x7f17b8101970 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.061+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b806a110 con 0x7f17b8101970 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.062+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17a8022590 con 0x7f17b8101970 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.062+0000 7f17bc9e2700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 0x7f17a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.062+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f17a80d14d0 con 0x7f17b8101970 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.062+0000 7f17b67fc700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 0x7f17a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:42.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.063+0000 7f17b67fc700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 0x7f17a00a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:42.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.066+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17a8030070 con 0x7f17b8101970 2024-04-26T18:21:42.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:41 smithi037 ceph-mon[29171]: pgmap v310: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:42.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:41 smithi120 ceph-mon[32244]: pgmap v310: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:42.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.306+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f17b8063860 con 0x7f17a00a3cd0 2024-04-26T18:21:42.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:41 smithi170 ceph-mon[32697]: pgmap v310: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:21:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.336+0000 7f17bc9e2700 1 -- 172.21.15.37:0/2227627453 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f17b8063860 con 0x7f17a00a3cd0 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 msgr2=0x7f17a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 0x7f17a00a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005950 tx=0x7f17ac0058e0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 msgr2=0x7f17b8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b580 tx=0x7f17a80047b0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 shutdown_connections 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f17a00a3cd0 0x7f17a00a6190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17b81094d0 0x7f17b81035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b8101970 0x7f17b8102b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 --2- 172.21.15.37:0/2227627453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17b8102330 0x7f17b8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 >> 172.21.15.37:0/2227627453 conn(0x7f17b8078920 msgr2=0x7f17b8100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 shutdown_connections 2024-04-26T18:21:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:42.338+0000 7f17bd9e4700 1 -- 172.21.15.37:0/2227627453 wait complete. 2024-04-26T18:21:42.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T18:21:43.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:43 smithi037 ceph-mon[29171]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:43.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:43 smithi120 ceph-mon[32244]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:43.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:43 smithi170 ceph-mon[32697]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:44.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:44 smithi120 ceph-mon[32244]: pgmap v311: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T18:21:44.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:44 smithi170 ceph-mon[32697]: pgmap v311: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T18:21:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:44 smithi037 ceph-mon[29171]: pgmap v311: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-04-26T18:21:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 -- 172.21.15.37:0/3530480646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006b130 msgr2=0x7f495006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 --2- 172.21.15.37:0/3530480646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006b130 0x7f495006b590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4944009990 tx=0x7f4944009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 -- 172.21.15.37:0/3530480646 shutdown_connections 2024-04-26T18:21:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 --2- 172.21.15.37:0/3530480646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006bad0 0x7f49501102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 --2- 172.21.15.37:0/3530480646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 0x7f4950109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 --2- 172.21.15.37:0/3530480646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006b130 0x7f495006b590 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 -- 172.21.15.37:0/3530480646 >> 172.21.15.37:0/3530480646 conn(0x7f49500792d0 msgr2=0x7f49500796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 -- 172.21.15.37:0/3530480646 shutdown_connections 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.981+0000 7f4957023700 1 -- 172.21.15.37:0/3530480646 wait complete. 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 Processor -- start 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 -- start start 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006b130 0x7f495019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 0x7f49501999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49501135b0 con 0x7f495006bad0 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4950113430 con 0x7f49501094a0 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4957023700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4950113730 con 0x7f495006b130 2024-04-26T18:21:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35440/0 (socket says 172.21.15.37:35440) 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 -- 172.21.15.37:0/1626381881 learned_addr learned my addr 172.21.15.37:0/1626381881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f4954dbf700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006b130 0x7f495019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f49555c0700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 0x7f49501999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 -- 172.21.15.37:0/1626381881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006b130 msgr2=0x7f495019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006b130 0x7f495019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 -- 172.21.15.37:0/1626381881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 msgr2=0x7f49501999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 0x7f49501999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.982+0000 7f494ffff700 1 -- 172.21.15.37:0/1626381881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495019a100 con 0x7f495006bad0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.983+0000 7f494ffff700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c00 tx=0x7f49440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.983+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944027070 con 0x7f495006bad0 2024-04-26T18:21:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.983+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495019a390 con 0x7f495006bad0 2024-04-26T18:21:44.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.983+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4944003f20 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.984+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944022c30 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.984+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49501a66b0 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.984+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495006a110 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.985+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f494402a070 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.986+0000 7f494dffb700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 0x7f49380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.986+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f49440d1280 con 0x7f495006bad0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.986+0000 7f4954dbf700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 0x7f49380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:44.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.986+0000 7f4954dbf700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 0x7f49380a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4948009550 tx=0x7f49480091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:44.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:44.991+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f494405eff0 con 0x7f495006bad0 2024-04-26T18:21:45.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.235+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f495019aed0 con 0x7f49380a3cd0 2024-04-26T18:21:45.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.240+0000 7f494dffb700 1 -- 172.21.15.37:0/1626381881 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f495019aed0 con 0x7f49380a3cd0 2024-04-26T18:21:45.241 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/44bc6169-0263-4180-8dc4-a35378b8a772 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.242+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 msgr2=0x7f49380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.242+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 0x7f49380a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4948009550 tx=0x7f49480091b0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.242+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 msgr2=0x7f495019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.242+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4944000c00 tx=0x7f49440047b0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 shutdown_connections 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49380a3cd0 0x7f49380a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49501094a0 0x7f49501999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495006b130 0x7f495019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 --2- 172.21.15.37:0/1626381881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f495006bad0 0x7f495019fd20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 >> 172.21.15.37:0/1626381881 conn(0x7f49500792d0 msgr2=0x7f49501004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 shutdown_connections 2024-04-26T18:21:45.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:45.243+0000 7f4957023700 1 -- 172.21.15.37:0/1626381881 wait complete. 2024-04-26T18:21:45.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:21:46.114 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:46.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:46 smithi037 ceph-mon[29171]: pgmap v312: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-26T18:21:46.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:46 smithi037 ceph-mon[29171]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:46 smithi170 ceph-mon[32697]: pgmap v312: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-26T18:21:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:46 smithi170 ceph-mon[32697]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:46.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:46 smithi120 ceph-mon[32244]: pgmap v312: 97 pgs: 97 active+clean; 602 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-26T18:21:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:46 smithi120 ceph-mon[32244]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 -- 172.21.15.37:0/1352296807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d041094e0 msgr2=0x7f8d041098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/1352296807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d041094e0 0x7f8d041098c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009990 tx=0x7f8cf8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 -- 172.21.15.37:0/1352296807 shutdown_connections 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/1352296807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04103e60 0x7f8d041102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/1352296807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d041094e0 0x7f8d041098c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/1352296807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 -- 172.21.15.37:0/1352296807 >> 172.21.15.37:0/1352296807 conn(0x7f8d0407a590 msgr2=0x7f8d0407a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 -- 172.21.15.37:0/1352296807 shutdown_connections 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.806+0000 7f8d0bb96700 1 -- 172.21.15.37:0/1352296807 wait complete. 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 Processor -- start 2024-04-26T18:21:47.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 -- start start 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d041094e0 0x7f8d040761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8d04113660 con 0x7f8d04103e60 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041134e0 con 0x7f8d041034a0 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d0bb96700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041137e0 con 0x7f8d041094e0 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d09131700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d09131700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35462/0 (socket says 172.21.15.37:35462) 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.807+0000 7f8d09131700 1 -- 172.21.15.37:0/2100400400 learned_addr learned my addr 172.21.15.37:0/2100400400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09932700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d0a133700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d041094e0 0x7f8d040761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:47.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 -- 172.21.15.37:0/2100400400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d041094e0 msgr2=0x7f8d040761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d041094e0 0x7f8d040761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 -- 172.21.15.37:0/2100400400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 msgr2=0x7f8d04075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 -- 172.21.15.37:0/2100400400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d041a80a0 con 0x7f8d04103e60 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09932700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d09131700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d950 tx=0x7f8d0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00016070 con 0x7f8d04103e60 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d041a8390 con 0x7f8d04103e60 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d00010460 con 0x7f8d04103e60 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0000f620 con 0x7f8d04103e60 2024-04-26T18:21:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.808+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a8920 con 0x7f8d04103e60 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.809+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0410b950 con 0x7f8d04103e60 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.810+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d00019070 con 0x7f8d04103e60 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.811+0000 7f8cf6ffd700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 0x7f8cf00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.811+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8d000d0340 con 0x7f8d04103e60 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.811+0000 7f8d09932700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 0x7f8cf00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.811+0000 7f8d09932700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 0x7f8cf00a6210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800b5c0 tx=0x7f8cf8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:47.816+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d0009cd70 con 0x7f8d04103e60 2024-04-26T18:21:48.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.077+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d0406a110 con 0x7f8d04103e60 2024-04-26T18:21:48.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:48 smithi037 ceph-mon[29171]: pgmap v313: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-26T18:21:48.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.077+0000 7f8cf6ffd700 1 -- 172.21.15.37:0/2100400400 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7f8d0009c670 con 0x7f8d04103e60 2024-04-26T18:21:48.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:48.078 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.079+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 msgr2=0x7f8cf00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.079+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 0x7f8cf00a6210 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800b5c0 tx=0x7f8cf8005fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.079+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 msgr2=0x7f8d04075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.079+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d950 tx=0x7f8d0000dd10 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 shutdown_connections 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8cf00a3d50 0x7f8cf00a6210 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d041094e0 0x7f8d040761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d041034a0 0x7f8d04075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 --2- 172.21.15.37:0/2100400400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d04103e60 0x7f8d04075c60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 >> 172.21.15.37:0/2100400400 conn(0x7f8d0407a590 msgr2=0x7f8d041021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 shutdown_connections 2024-04-26T18:21:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:48.080+0000 7f8d0bb96700 1 -- 172.21.15.37:0/2100400400 wait complete. 2024-04-26T18:21:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:21:48.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:48 smithi170 ceph-mon[32697]: pgmap v313: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-26T18:21:48.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:48 smithi120 ceph-mon[32244]: pgmap v313: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-04-26T18:21:48.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:21:49.013 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:49.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:49 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2100400400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:21:49.340 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:49 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2100400400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:21:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:49 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2100400400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:21:50.112 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:50 smithi037 ceph-mon[29171]: pgmap v314: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T18:21:50.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:50 smithi170 ceph-mon[32697]: pgmap v314: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T18:21:50.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:50 smithi120 ceph-mon[32244]: pgmap v314: 97 pgs: 97 active+clean; 622 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 -- 172.21.15.37:0/1517112762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f433406c590 msgr2=0x7f4334110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 --2- 172.21.15.37:0/1517112762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f433406c590 0x7f4334110310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b0a0 tx=0x7f432c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 -- 172.21.15.37:0/1517112762 shutdown_connections 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 --2- 172.21.15.37:0/1517112762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f433406c590 0x7f4334110310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 --2- 172.21.15.37:0/1517112762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43341094d0 0x7f43341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 --2- 172.21.15.37:0/1517112762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 0x7f433406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.647+0000 7f4339515700 1 -- 172.21.15.37:0/1517112762 >> 172.21.15.37:0/1517112762 conn(0x7f43340793c0 msgr2=0x7f43340797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.648+0000 7f4339515700 1 -- 172.21.15.37:0/1517112762 shutdown_connections 2024-04-26T18:21:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.648+0000 7f4339515700 1 -- 172.21.15.37:0/1517112762 wait complete. 2024-04-26T18:21:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.648+0000 7f4339515700 1 Processor -- start 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.648+0000 7f4339515700 1 -- start start 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 0x7f4334102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f433406c590 0x7f43341028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4334113610 con 0x7f43341094d0 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4334113490 con 0x7f433406c590 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4334113790 con 0x7f433406bbd0 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35478/0 (socket says 172.21.15.37:35478) 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 -- 172.21.15.37:0/151163096 learned_addr learned my addr 172.21.15.37:0/151163096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4332ffd700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 0x7f4334102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43327fc700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f433406c590 0x7f43341028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 -- 172.21.15.37:0/151163096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 msgr2=0x7f4334102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 0x7f4334102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 -- 172.21.15.37:0/151163096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f433406c590 msgr2=0x7f43341028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f433406c590 0x7f43341028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 -- 172.21.15.37:0/151163096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43341a8000 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f43337fe700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b830 tx=0x7f432c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c021070 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f432c007e30 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43341a8290 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c004700 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.649+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a86f0 con 0x7f43341094d0 2024-04-26T18:21:50.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.650+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433410b9b0 con 0x7f43341094d0 2024-04-26T18:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.651+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f432c003d70 con 0x7f43341094d0 2024-04-26T18:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.652+0000 7f431bfff700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 0x7f431c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:50.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.652+0000 7f4332ffd700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 0x7f431c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.652+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f432c0d13a0 con 0x7f43341094d0 2024-04-26T18:21:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.653+0000 7f4332ffd700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 0x7f431c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4324007b60 tx=0x7f43240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:50.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.656+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f432c09de60 con 0x7f43341094d0 2024-04-26T18:21:50.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.864+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f433402e490 con 0x7f43341094d0 2024-04-26T18:21:50.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.866+0000 7f431bfff700 1 -- 172.21.15.37:0/151163096 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22179 (secure 0 0 0) 0x7f432c09d760 con 0x7f43341094d0 2024-04-26T18:21:50.866 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:50.866 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:21:10.996894+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:21:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 msgr2=0x7f431c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 0x7f431c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4324007b60 tx=0x7f43240073d0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 msgr2=0x7f4334102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b830 tx=0x7f432c004ce0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 shutdown_connections 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f431c0a3cd0 0x7f431c0a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43341094d0 0x7f4334102e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f433406bbd0 0x7f4334102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 --2- 172.21.15.37:0/151163096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f433406c590 0x7f43341028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 >> 172.21.15.37:0/151163096 conn(0x7f43340793c0 msgr2=0x7f43341003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 shutdown_connections 2024-04-26T18:21:50.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:50.868+0000 7f4339515700 1 -- 172.21.15.37:0/151163096 wait complete. 2024-04-26T18:21:51.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:51 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/151163096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:51.317 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:51 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/151163096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:51.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:51 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/151163096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:21:51.695 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:21:51.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:21:51.948 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:52.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:52 smithi037 ceph-mon[29171]: pgmap v315: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:21:52.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:52 smithi170 ceph-mon[32697]: pgmap v315: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:21:52.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:52 smithi120 ceph-mon[32244]: pgmap v315: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- 172.21.15.37:0/1332399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f126410d990 msgr2=0x7f126410fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1332399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f126410d990 0x7f126410fe00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b0a0 tx=0x7f125c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- 172.21.15.37:0/1332399769 shutdown_connections 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1332399769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f126410d990 0x7f126410fe00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1332399769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12641045b0 0x7f1264104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1332399769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1264104ed0 0x7f126410d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- 172.21.15.37:0/1332399769 >> 172.21.15.37:0/1332399769 conn(0x7f12640fdd40 msgr2=0x7f1264100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- 172.21.15.37:0/1332399769 shutdown_connections 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- 172.21.15.37:0/1332399769 wait complete. 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 Processor -- start 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.542+0000 7f126a1a0700 1 -- start start 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12641045b0 0x7f1264102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126410d990 0x7f1264102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12641135b0 con 0x7f1264104ed0 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1264113430 con 0x7f126410d990 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126a1a0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1264113730 con 0x7f12641045b0 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35506/0 (socket says 172.21.15.37:35506) 2024-04-26T18:21:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 -- 172.21.15.37:0/1986344340 learned_addr learned my addr 172.21.15.37:0/1986344340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126919e700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12641045b0 0x7f1264102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126999f700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126410d990 0x7f1264102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 -- 172.21.15.37:0/1986344340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12641045b0 msgr2=0x7f1264102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12641045b0 0x7f1264102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 -- 172.21.15.37:0/1986344340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126410d990 msgr2=0x7f1264102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126410d990 0x7f1264102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 -- 172.21.15.37:0/1986344340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641a7fd0 con 0x7f1264104ed0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.543+0000 7f126899d700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f125400d950 tx=0x7f125400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254017070 con 0x7f1264104ed0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1254005020 con 0x7f1264104ed0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400fa10 con 0x7f1264104ed0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641a82c0 con 0x7f1264104ed0 2024-04-26T18:21:53.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641a87b0 con 0x7f1264104ed0 2024-04-26T18:21:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.544+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126406a110 con 0x7f1264104ed0 2024-04-26T18:21:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.546+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1254004a30 con 0x7f1264104ed0 2024-04-26T18:21:53.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.546+0000 7f125a7fc700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 0x7f12500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.546+0000 7f126919e700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 0x7f12500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.546+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f12540cff80 con 0x7f1264104ed0 2024-04-26T18:21:53.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.547+0000 7f126919e700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 0x7f12500a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1260005950 tx=0x7f126000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:53.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.550+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f125409c9b0 con 0x7f1264104ed0 2024-04-26T18:21:53.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.751+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1264063860 con 0x7f12500a3cd0 2024-04-26T18:21:53.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.759+0000 7f125a7fc700 1 -- 172.21.15.37:0/1986344340 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258708 (secure 0 0 0) 0x7f1264063860 con 0x7f12500a3cd0 2024-04-26T18:21:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 msgr2=0x7f12500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 0x7f12500a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1260005950 tx=0x7f126000b410 comp rx=0 tx=0).stop 2024-04-26T18:21:53.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 msgr2=0x7f1264102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f125400d950 tx=0x7f125400dc60 comp rx=0 tx=0).stop 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 shutdown_connections 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f12500a3cd0 0x7f12500a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f126410d990 0x7f1264102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12641045b0 0x7f1264102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 --2- 172.21.15.37:0/1986344340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1264104ed0 0x7f1264102750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.763+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 >> 172.21.15.37:0/1986344340 conn(0x7f12640fdd40 msgr2=0x7f12640feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.764+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 shutdown_connections 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:53.764+0000 7f126a1a0700 1 -- 172.21.15.37:0/1986344340 wait complete. 2024-04-26T18:21:53.767 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:21:54.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:54 smithi170 ceph-mon[32697]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:54.409 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:54 smithi037 ceph-mon[29171]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:54.437 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-04-26T18:21:52.682561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649555,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":305,"num_read_kb":266,"num_write":342,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383248,"kb_used_data":13296,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558576,"statfs":{"total":1151940427776,"available":1151547981824,"internally_reserved":0,"allocated":13615104,"data_stored":7496263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":22,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901365+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:21:11.901365+0000","last_peered":"2024-04-26T18:21:11.901365+0000","last_clean":"2024-04-26T18:21:11.901365+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:21:11.901365+0000","last_undegraded":"2024-04-26T18:21:11.901365+0000","last_fullsized":"2024-04-26T18:21:11.901365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917530+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:21:11.917530+0000","last_peered":"2024-04-26T18:21:11.917530+0000","last_clean":"2024-04-26T18:21:11.917530+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:21:11.917530+0000","last_undegraded":"2024-04-26T18:21:11.917530+0000","last_fullsized":"2024-04-26T18:21:11.917530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917156+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:21:11.917156+0000","last_peered":"2024-04-26T18:21:11.917156+0000","last_clean":"2024-04-26T18:21:11.917156+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:21:11.917156+0000","last_undegraded":"2024-04-26T18:21:11.917156+0000","last_fullsized":"2024-04-26T18:21:11.917156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912775+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:21:11.912775+0000","last_peered":"2024-04-26T18:21:11.912775+0000","last_clean":"2024-04-26T18:21:11.912775+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:21:11.912775+0000","last_undegraded":"2024-04-26T18:21:11.912775+0000","last_fullsized":"2024-04-26T18:21:11.912775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913458+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:21:11.913458+0000","last_peered":"2024-04-26T18:21:11.913458+0000","last_clean":"2024-04-26T18:21:11.913458+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:21:11.913458+0000","last_undegraded":"2024-04-26T18:21:11.913458+0000","last_fullsized":"2024-04-26T18:21:11.913458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900548+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:21:11.900548+0000","last_peered":"2024-04-26T18:21:11.900548+0000","last_clean":"2024-04-26T18:21:11.900548+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:21:11.900548+0000","last_undegraded":"2024-04-26T18:21:11.900548+0000","last_fullsized":"2024-04-26T18:21:11.900548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911559+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:21:11.911559+0000","last_peered":"2024-04-26T18:21:11.911559+0000","last_clean":"2024-04-26T18:21:11.911559+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:21:11.911559+0000","last_undegraded":"2024-04-26T18:21:11.911559+0000","last_fullsized":"2024-04-26T18:21:11.911559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914698+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:21:11.914698+0000","last_peered":"2024-04-26T18:21:11.914698+0000","last_clean":"2024-04-26T18:21:11.914698+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:21:11.914698+0000","last_undegraded":"2024-04-26T18:21:11.914698+0000","last_fullsized":"2024-04-26T18:21:11.914698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914292+0000","last_change":"2024-04-26T18:21:08.902438+0000","last_active":"2024-04-26T18:21:11.914292+0000","last_peered":"2024-04-26T18:21:11.914292+0000","last_clean":"2024-04-26T18:21:11.914292+0000","last_became_active":"2024-04-26T18:21:08.902094+0000","last_became_peered":"2024-04-26T18:21:08.902094+0000","last_unstale":"2024-04-26T18:21:11.914292+0000","last_undegraded":"2024-04-26T18:21:11.914292+0000","last_fullsized":"2024-04-26T18:21:11.914292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:03.731129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900947+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:21:11.900947+0000","last_peered":"2024-04-26T18:21:11.900947+0000","last_clean":"2024-04-26T18:21:11.900947+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:21:11.900947+0000","last_undegraded":"2024-04-26T18:21:11.900947+0000","last_fullsized":"2024-04-26T18:21:11.900947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913122+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:21:11.913122+0000","last_peered":"2024-04-26T18:21:11.913122+0000","last_clean":"2024-04-26T18:21:11.913122+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:21:11.913122+0000","last_undegraded":"2024-04-26T18:21:11.913122+0000","last_fullsized":"2024-04-26T18:21:11.913122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912635+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:21:11.912635+0000","last_peered":"2024-04-26T18:21:11.912635+0000","last_clean":"2024-04-26T18:21:11.912635+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:21:11.912635+0000","last_undegraded":"2024-04-26T18:21:11.912635+0000","last_fullsized":"2024-04-26T18:21:11.912635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912618+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:21:11.912618+0000","last_peered":"2024-04-26T18:21:11.912618+0000","last_clean":"2024-04-26T18:21:11.912618+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:21:11.912618+0000","last_undegraded":"2024-04-26T18:21:11.912618+0000","last_fullsized":"2024-04-26T18:21:11.912618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916940+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:21:11.916940+0000","last_peered":"2024-04-26T18:21:11.916940+0000","last_clean":"2024-04-26T18:21:11.916940+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:21:11.916940+0000","last_undegraded":"2024-04-26T18:21:11.916940+0000","last_fullsized":"2024-04-26T18:21:11.916940+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916708+0000","last_change":"2024-04-26T18:21:08.912832+0000","last_active":"2024-04-26T18:21:11.916708+0000","last_peered":"2024-04-26T18:21:11.916708+0000","last_clean":"2024-04-26T18:21:11.916708+0000","last_became_active":"2024-04-26T18:21:08.912269+0000","last_became_peered":"2024-04-26T18:21:08.912269+0000","last_unstale":"2024-04-26T18:21:11.916708+0000","last_undegraded":"2024-04-26T18:21:11.916708+0000","last_fullsized":"2024-04-26T18:21:11.916708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:37:15.142761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913470+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:21:11.913470+0000","last_peered":"2024-04-26T18:21:11.913470+0000","last_clean":"2024-04-26T18:21:11.913470+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:21:11.913470+0000","last_undegraded":"2024-04-26T18:21:11.913470+0000","last_fullsized":"2024-04-26T18:21:11.913470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901484+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:21:11.901484+0000","last_peered":"2024-04-26T18:21:11.901484+0000","last_clean":"2024-04-26T18:21:11.901484+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:21:11.901484+0000","last_undegraded":"2024-04-26T18:21:11.901484+0000","last_fullsized":"2024-04-26T18:21:11.901484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901347+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:21:11.901347+0000","last_peered":"2024-04-26T18:21:11.901347+0000","last_clean":"2024-04-26T18:21:11.901347+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:21:11.901347+0000","last_undegraded":"2024-04-26T18:21:11.901347+0000","last_fullsized":"2024-04-26T18:21:11.901347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914125+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:21:11.914125+0000","last_peered":"2024-04-26T18:21:11.914125+0000","last_clean":"2024-04-26T18:21:11.914125+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:21:11.914125+0000","last_undegraded":"2024-04-26T18:21:11.914125+0000","last_fullsized":"2024-04-26T18:21:11.914125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950922+0000","last_change":"2024-04-26T18:21:05.874286+0000","last_active":"2024-04-26T18:21:24.950922+0000","last_peered":"2024-04-26T18:21:24.950922+0000","last_clean":"2024-04-26T18:21:24.950922+0000","last_became_active":"2024-04-26T18:21:05.873882+0000","last_became_peered":"2024-04-26T18:21:05.873882+0000","last_unstale":"2024-04-26T18:21:24.950922+0000","last_undegraded":"2024-04-26T18:21:24.950922+0000","last_fullsized":"2024-04-26T18:21:24.950922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:26.225532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915966+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:21:11.915966+0000","last_peered":"2024-04-26T18:21:11.915966+0000","last_clean":"2024-04-26T18:21:11.915966+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:21:11.915966+0000","last_undegraded":"2024-04-26T18:21:11.915966+0000","last_fullsized":"2024-04-26T18:21:11.915966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918652+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:21:11.918652+0000","last_peered":"2024-04-26T18:21:11.918652+0000","last_clean":"2024-04-26T18:21:11.918652+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:21:11.918652+0000","last_undegraded":"2024-04-26T18:21:11.918652+0000","last_fullsized":"2024-04-26T18:21:11.918652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917725+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:21:11.917725+0000","last_peered":"2024-04-26T18:21:11.917725+0000","last_clean":"2024-04-26T18:21:11.917725+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:21:11.917725+0000","last_undegraded":"2024-04-26T18:21:11.917725+0000","last_fullsized":"2024-04-26T18:21:11.917725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950231+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:21:24.950231+0000","last_peered":"2024-04-26T18:21:24.950231+0000","last_clean":"2024-04-26T18:21:24.950231+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:21:24.950231+0000","last_undegraded":"2024-04-26T18:21:24.950231+0000","last_fullsized":"2024-04-26T18:21:24.950231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913425+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:21:11.913425+0000","last_peered":"2024-04-26T18:21:11.913425+0000","last_clean":"2024-04-26T18:21:11.913425+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:21:11.913425+0000","last_undegraded":"2024-04-26T18:21:11.913425+0000","last_fullsized":"2024-04-26T18:21:11.913425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.919441+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:21:11.919441+0000","last_peered":"2024-04-26T18:21:11.919441+0000","last_clean":"2024-04-26T18:21:11.919441+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:21:11.919441+0000","last_undegraded":"2024-04-26T18:21:11.919441+0000","last_fullsized":"2024-04-26T18:21:11.919441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913206+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:21:11.913206+0000","last_peered":"2024-04-26T18:21:11.913206+0000","last_clean":"2024-04-26T18:21:11.913206+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:21:11.913206+0000","last_undegraded":"2024-04-26T18:21:11.913206+0000","last_fullsized":"2024-04-26T18:21:11.913206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:47.988014+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:21:47.988014+0000","last_peered":"2024-04-26T18:21:47.988014+0000","last_clean":"2024-04-26T18:21:47.988014+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:21:47.988014+0000","last_undegraded":"2024-04-26T18:21:47.988014+0000","last_fullsized":"2024-04-26T18:21:47.988014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915281+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:21:11.915281+0000","last_peered":"2024-04-26T18:21:11.915281+0000","last_clean":"2024-04-26T18:21:11.915281+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:21:11.915281+0000","last_undegraded":"2024-04-26T18:21:11.915281+0000","last_fullsized":"2024-04-26T18:21:11.915281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.948964+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:21:24.948964+0000","last_peered":"2024-04-26T18:21:24.948964+0000","last_clean":"2024-04-26T18:21:24.948964+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:21:24.948964+0000","last_undegraded":"2024-04-26T18:21:24.948964+0000","last_fullsized":"2024-04-26T18:21:24.948964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912954+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:21:11.912954+0000","last_peered":"2024-04-26T18:21:11.912954+0000","last_clean":"2024-04-26T18:21:11.912954+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:21:11.912954+0000","last_undegraded":"2024-04-26T18:21:11.912954+0000","last_fullsized":"2024-04-26T18:21:11.912954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949265+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:21:24.949265+0000","last_peered":"2024-04-26T18:21:24.949265+0000","last_clean":"2024-04-26T18:21:24.949265+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:21:24.949265+0000","last_undegraded":"2024-04-26T18:21:24.949265+0000","last_fullsized":"2024-04-26T18:21:24.949265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914789+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:21:11.914789+0000","last_peered":"2024-04-26T18:21:11.914789+0000","last_clean":"2024-04-26T18:21:11.914789+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:21:11.914789+0000","last_undegraded":"2024-04-26T18:21:11.914789+0000","last_fullsized":"2024-04-26T18:21:11.914789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913595+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:21:11.913595+0000","last_peered":"2024-04-26T18:21:11.913595+0000","last_clean":"2024-04-26T18:21:11.913595+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:21:11.913595+0000","last_undegraded":"2024-04-26T18:21:11.913595+0000","last_fullsized":"2024-04-26T18:21:11.913595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901023+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:21:11.901023+0000","last_peered":"2024-04-26T18:21:11.901023+0000","last_clean":"2024-04-26T18:21:11.901023+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:21:11.901023+0000","last_undegraded":"2024-04-26T18:21:11.901023+0000","last_fullsized":"2024-04-26T18:21:11.901023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.988880+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:21:48.988880+0000","last_peered":"2024-04-26T18:21:48.988880+0000","last_clean":"2024-04-26T18:21:48.988880+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:21:48.988880+0000","last_undegraded":"2024-04-26T18:21:48.988880+0000","last_fullsized":"2024-04-26T18:21:48.988880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914004+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:21:11.914004+0000","last_peered":"2024-04-26T18:21:11.914004+0000","last_clean":"2024-04-26T18:21:11.914004+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:21:11.914004+0000","last_undegraded":"2024-04-26T18:21:11.914004+0000","last_fullsized":"2024-04-26T18:21:11.914004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941023+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:21:23.941023+0000","last_peered":"2024-04-26T18:21:23.941023+0000","last_clean":"2024-04-26T18:21:23.941023+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:21:23.941023+0000","last_undegraded":"2024-04-26T18:21:23.941023+0000","last_fullsized":"2024-04-26T18:21:23.941023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917530+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:21:11.917530+0000","last_peered":"2024-04-26T18:21:11.917530+0000","last_clean":"2024-04-26T18:21:11.917530+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:21:11.917530+0000","last_undegraded":"2024-04-26T18:21:11.917530+0000","last_fullsized":"2024-04-26T18:21:11.917530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.942257+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:21:23.942257+0000","last_peered":"2024-04-26T18:21:23.942257+0000","last_clean":"2024-04-26T18:21:23.942257+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:21:23.942257+0000","last_undegraded":"2024-04-26T18:21:23.942257+0000","last_fullsized":"2024-04-26T18:21:23.942257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913605+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:21:11.913605+0000","last_peered":"2024-04-26T18:21:11.913605+0000","last_clean":"2024-04-26T18:21:11.913605+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:21:11.913605+0000","last_undegraded":"2024-04-26T18:21:11.913605+0000","last_fullsized":"2024-04-26T18:21:11.913605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'25","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:49.991397+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:21:49.991397+0000","last_peered":"2024-04-26T18:21:49.991397+0000","last_clean":"2024-04-26T18:21:49.991397+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:21:49.991397+0000","last_undegraded":"2024-04-26T18:21:49.991397+0000","last_fullsized":"2024-04-26T18:21:49.991397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":55,"num_read_kb":80,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912144+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:21:11.912144+0000","last_peered":"2024-04-26T18:21:11.912144+0000","last_clean":"2024-04-26T18:21:11.912144+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:21:11.912144+0000","last_undegraded":"2024-04-26T18:21:11.912144+0000","last_fullsized":"2024-04-26T18:21:11.912144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949570+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:21:24.949570+0000","last_peered":"2024-04-26T18:21:24.949570+0000","last_clean":"2024-04-26T18:21:24.949570+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:21:24.949570+0000","last_undegraded":"2024-04-26T18:21:24.949570+0000","last_fullsized":"2024-04-26T18:21:24.949570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912116+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:21:11.912116+0000","last_peered":"2024-04-26T18:21:11.912116+0000","last_clean":"2024-04-26T18:21:11.912116+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:21:11.912116+0000","last_undegraded":"2024-04-26T18:21:11.912116+0000","last_fullsized":"2024-04-26T18:21:11.912116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941380+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:21:23.941380+0000","last_peered":"2024-04-26T18:21:23.941380+0000","last_clean":"2024-04-26T18:21:23.941380+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:21:23.941380+0000","last_undegraded":"2024-04-26T18:21:23.941380+0000","last_fullsized":"2024-04-26T18:21:23.941380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913091+0000","last_change":"2024-04-26T18:21:08.911668+0000","last_active":"2024-04-26T18:21:11.913091+0000","last_peered":"2024-04-26T18:21:11.913091+0000","last_clean":"2024-04-26T18:21:11.913091+0000","last_became_active":"2024-04-26T18:21:08.911321+0000","last_became_peered":"2024-04-26T18:21:08.911321+0000","last_unstale":"2024-04-26T18:21:11.913091+0000","last_undegraded":"2024-04-26T18:21:11.913091+0000","last_fullsized":"2024-04-26T18:21:11.913091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.457356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.989810+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:21:48.989810+0000","last_peered":"2024-04-26T18:21:48.989810+0000","last_clean":"2024-04-26T18:21:48.989810+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:21:48.989810+0000","last_undegraded":"2024-04-26T18:21:48.989810+0000","last_fullsized":"2024-04-26T18:21:48.989810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911387+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:21:11.911387+0000","last_peered":"2024-04-26T18:21:11.911387+0000","last_clean":"2024-04-26T18:21:11.911387+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:21:11.911387+0000","last_undegraded":"2024-04-26T18:21:11.911387+0000","last_fullsized":"2024-04-26T18:21:11.911387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950989+0000","last_change":"2024-04-26T18:21:06.144981+0000","last_active":"2024-04-26T18:21:24.950989+0000","last_peered":"2024-04-26T18:21:24.950989+0000","last_clean":"2024-04-26T18:21:24.950989+0000","last_became_active":"2024-04-26T18:21:06.144573+0000","last_became_peered":"2024-04-26T18:21:06.144573+0000","last_unstale":"2024-04-26T18:21:24.950989+0000","last_undegraded":"2024-04-26T18:21:24.950989+0000","last_fullsized":"2024-04-26T18:21:24.950989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:43.182801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912078+0000","last_change":"2024-04-26T18:21:08.899461+0000","last_active":"2024-04-26T18:21:11.912078+0000","last_peered":"2024-04-26T18:21:11.912078+0000","last_clean":"2024-04-26T18:21:11.912078+0000","last_became_active":"2024-04-26T18:21:08.899044+0000","last_became_peered":"2024-04-26T18:21:08.899044+0000","last_unstale":"2024-04-26T18:21:11.912078+0000","last_undegraded":"2024-04-26T18:21:11.912078+0000","last_fullsized":"2024-04-26T18:21:11.912078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:20.427553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:25.946239+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:21:25.946239+0000","last_peered":"2024-04-26T18:21:25.946239+0000","last_clean":"2024-04-26T18:21:25.946239+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:21:25.946239+0000","last_undegraded":"2024-04-26T18:21:25.946239+0000","last_fullsized":"2024-04-26T18:21:25.946239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901349+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:21:11.901349+0000","last_peered":"2024-04-26T18:21:11.901349+0000","last_clean":"2024-04-26T18:21:11.901349+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:21:11.901349+0000","last_undegraded":"2024-04-26T18:21:11.901349+0000","last_fullsized":"2024-04-26T18:21:11.901349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901696+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:21:11.901696+0000","last_peered":"2024-04-26T18:21:11.901696+0000","last_clean":"2024-04-26T18:21:11.901696+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:21:11.901696+0000","last_undegraded":"2024-04-26T18:21:11.901696+0000","last_fullsized":"2024-04-26T18:21:11.901696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900768+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:21:11.900768+0000","last_peered":"2024-04-26T18:21:11.900768+0000","last_clean":"2024-04-26T18:21:11.900768+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:21:11.900768+0000","last_undegraded":"2024-04-26T18:21:11.900768+0000","last_fullsized":"2024-04-26T18:21:11.900768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911960+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:21:11.911960+0000","last_peered":"2024-04-26T18:21:11.911960+0000","last_clean":"2024-04-26T18:21:11.911960+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:21:11.911960+0000","last_undegraded":"2024-04-26T18:21:11.911960+0000","last_fullsized":"2024-04-26T18:21:11.911960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900592+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:21:11.900592+0000","last_peered":"2024-04-26T18:21:11.900592+0000","last_clean":"2024-04-26T18:21:11.900592+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:21:11.900592+0000","last_undegraded":"2024-04-26T18:21:11.900592+0000","last_fullsized":"2024-04-26T18:21:11.900592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912720+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:21:11.912720+0000","last_peered":"2024-04-26T18:21:11.912720+0000","last_clean":"2024-04-26T18:21:11.912720+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:21:11.912720+0000","last_undegraded":"2024-04-26T18:21:11.912720+0000","last_fullsized":"2024-04-26T18:21:11.912720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.989215+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:21:48.989215+0000","last_peered":"2024-04-26T18:21:48.989215+0000","last_clean":"2024-04-26T18:21:48.989215+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:21:48.989215+0000","last_undegraded":"2024-04-26T18:21:48.989215+0000","last_fullsized":"2024-04-26T18:21:48.989215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915311+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:21:11.915311+0000","last_peered":"2024-04-26T18:21:11.915311+0000","last_clean":"2024-04-26T18:21:11.915311+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:21:11.915311+0000","last_undegraded":"2024-04-26T18:21:11.915311+0000","last_fullsized":"2024-04-26T18:21:11.915311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:26.954972+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:21:26.954972+0000","last_peered":"2024-04-26T18:21:26.954972+0000","last_clean":"2024-04-26T18:21:26.954972+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:21:26.954972+0000","last_undegraded":"2024-04-26T18:21:26.954972+0000","last_fullsized":"2024-04-26T18:21:26.954972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911616+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:21:11.911616+0000","last_peered":"2024-04-26T18:21:11.911616+0000","last_clean":"2024-04-26T18:21:11.911616+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:21:11.911616+0000","last_undegraded":"2024-04-26T18:21:11.911616+0000","last_fullsized":"2024-04-26T18:21:11.911616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912234+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:21:11.912234+0000","last_peered":"2024-04-26T18:21:11.912234+0000","last_clean":"2024-04-26T18:21:11.912234+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:21:11.912234+0000","last_undegraded":"2024-04-26T18:21:11.912234+0000","last_fullsized":"2024-04-26T18:21:11.912234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949404+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:21:24.949404+0000","last_peered":"2024-04-26T18:21:24.949404+0000","last_clean":"2024-04-26T18:21:24.949404+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:21:24.949404+0000","last_undegraded":"2024-04-26T18:21:24.949404+0000","last_fullsized":"2024-04-26T18:21:24.949404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.939822+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:21:23.939822+0000","last_peered":"2024-04-26T18:21:23.939822+0000","last_clean":"2024-04-26T18:21:23.939822+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:21:23.939822+0000","last_undegraded":"2024-04-26T18:21:23.939822+0000","last_fullsized":"2024-04-26T18:21:23.939822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912127+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:21:11.912127+0000","last_peered":"2024-04-26T18:21:11.912127+0000","last_clean":"2024-04-26T18:21:11.912127+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:21:11.912127+0000","last_undegraded":"2024-04-26T18:21:11.912127+0000","last_fullsized":"2024-04-26T18:21:11.912127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:45.915663+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:21:45.915663+0000","last_peered":"2024-04-26T18:21:45.915663+0000","last_clean":"2024-04-26T18:21:45.915663+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:21:45.915663+0000","last_undegraded":"2024-04-26T18:21:45.915663+0000","last_fullsized":"2024-04-26T18:21:45.915663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916155+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:21:11.916155+0000","last_peered":"2024-04-26T18:21:11.916155+0000","last_clean":"2024-04-26T18:21:11.916155+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:21:11.916155+0000","last_undegraded":"2024-04-26T18:21:11.916155+0000","last_fullsized":"2024-04-26T18:21:11.916155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.940910+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:21:23.940910+0000","last_peered":"2024-04-26T18:21:23.940910+0000","last_clean":"2024-04-26T18:21:23.940910+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:21:23.940910+0000","last_undegraded":"2024-04-26T18:21:23.940910+0000","last_fullsized":"2024-04-26T18:21:23.940910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917651+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:21:11.917651+0000","last_peered":"2024-04-26T18:21:11.917651+0000","last_clean":"2024-04-26T18:21:11.917651+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:21:11.917651+0000","last_undegraded":"2024-04-26T18:21:11.917651+0000","last_fullsized":"2024-04-26T18:21:11.917651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.939922+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:21:23.939922+0000","last_peered":"2024-04-26T18:21:23.939922+0000","last_clean":"2024-04-26T18:21:23.939922+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:21:23.939922+0000","last_undegraded":"2024-04-26T18:21:23.939922+0000","last_fullsized":"2024-04-26T18:21:23.939922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911948+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:21:11.911948+0000","last_peered":"2024-04-26T18:21:11.911948+0000","last_clean":"2024-04-26T18:21:11.911948+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:21:11.911948+0000","last_undegraded":"2024-04-26T18:21:11.911948+0000","last_fullsized":"2024-04-26T18:21:11.911948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941723+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:21:23.941723+0000","last_peered":"2024-04-26T18:21:23.941723+0000","last_clean":"2024-04-26T18:21:23.941723+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:21:23.941723+0000","last_undegraded":"2024-04-26T18:21:23.941723+0000","last_fullsized":"2024-04-26T18:21:23.941723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900970+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:21:11.900970+0000","last_peered":"2024-04-26T18:21:11.900970+0000","last_clean":"2024-04-26T18:21:11.900970+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:21:11.900970+0000","last_undegraded":"2024-04-26T18:21:11.900970+0000","last_fullsized":"2024-04-26T18:21:11.900970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912237+0000","last_change":"2024-04-26T18:21:05.875607+0000","last_active":"2024-04-26T18:21:11.912237+0000","last_peered":"2024-04-26T18:21:11.912237+0000","last_clean":"2024-04-26T18:21:11.912237+0000","last_became_active":"2024-04-26T18:21:05.875255+0000","last_became_peered":"2024-04-26T18:21:05.875255+0000","last_unstale":"2024-04-26T18:21:11.912237+0000","last_undegraded":"2024-04-26T18:21:11.912237+0000","last_fullsized":"2024-04-26T18:21:11.912237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:47:28.674657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915267+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:21:11.915267+0000","last_peered":"2024-04-26T18:21:11.915267+0000","last_clean":"2024-04-26T18:21:11.915267+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:21:11.915267+0000","last_undegraded":"2024-04-26T18:21:11.915267+0000","last_fullsized":"2024-04-26T18:21:11.915267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941508+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:21:23.941508+0000","last_peered":"2024-04-26T18:21:23.941508+0000","last_clean":"2024-04-26T18:21:23.941508+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:21:23.941508+0000","last_undegraded":"2024-04-26T18:21:23.941508+0000","last_fullsized":"2024-04-26T18:21:23.941508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912076+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:21:11.912076+0000","last_peered":"2024-04-26T18:21:11.912076+0000","last_clean":"2024-04-26T18:21:11.912076+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:21:11.912076+0000","last_undegraded":"2024-04-26T18:21:11.912076+0000","last_fullsized":"2024-04-26T18:21:11.912076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.951329+0000","last_change":"2024-04-26T18:21:06.146648+0000","last_active":"2024-04-26T18:21:24.951329+0000","last_peered":"2024-04-26T18:21:24.951329+0000","last_clean":"2024-04-26T18:21:24.951329+0000","last_became_active":"2024-04-26T18:21:06.146016+0000","last_became_peered":"2024-04-26T18:21:06.146016+0000","last_unstale":"2024-04-26T18:21:24.951329+0000","last_undegraded":"2024-04-26T18:21:24.951329+0000","last_fullsized":"2024-04-26T18:21:24.951329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:52.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913404+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:21:11.913404+0000","last_peered":"2024-04-26T18:21:11.913404+0000","last_clean":"2024-04-26T18:21:11.913404+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:21:11.913404+0000","last_undegraded":"2024-04-26T18:21:11.913404+0000","last_fullsized":"2024-04-26T18:21:11.913404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918076+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:21:11.918076+0000","last_peered":"2024-04-26T18:21:11.918076+0000","last_clean":"2024-04-26T18:21:11.918076+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:21:11.918076+0000","last_undegraded":"2024-04-26T18:21:11.918076+0000","last_fullsized":"2024-04-26T18:21:11.918076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912466+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:21:11.912466+0000","last_peered":"2024-04-26T18:21:11.912466+0000","last_clean":"2024-04-26T18:21:11.912466+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:21:11.912466+0000","last_undegraded":"2024-04-26T18:21:11.912466+0000","last_fullsized":"2024-04-26T18:21:11.912466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917270+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:21:11.917270+0000","last_peered":"2024-04-26T18:21:11.917270+0000","last_clean":"2024-04-26T18:21:11.917270+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:21:11.917270+0000","last_undegraded":"2024-04-26T18:21:11.917270+0000","last_fullsized":"2024-04-26T18:21:11.917270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912825+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:21:11.912825+0000","last_peered":"2024-04-26T18:21:11.912825+0000","last_clean":"2024-04-26T18:21:11.912825+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:21:11.912825+0000","last_undegraded":"2024-04-26T18:21:11.912825+0000","last_fullsized":"2024-04-26T18:21:11.912825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901199+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:21:11.901199+0000","last_peered":"2024-04-26T18:21:11.901199+0000","last_clean":"2024-04-26T18:21:11.901199+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:21:11.901199+0000","last_undegraded":"2024-04-26T18:21:11.901199+0000","last_fullsized":"2024-04-26T18:21:11.901199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912357+0000","last_change":"2024-04-26T18:21:08.911306+0000","last_active":"2024-04-26T18:21:11.912357+0000","last_peered":"2024-04-26T18:21:11.912357+0000","last_clean":"2024-04-26T18:21:11.912357+0000","last_became_active":"2024-04-26T18:21:08.910942+0000","last_became_peered":"2024-04-26T18:21:08.910942+0000","last_unstale":"2024-04-26T18:21:11.912357+0000","last_undegraded":"2024-04-26T18:21:11.912357+0000","last_fullsized":"2024-04-26T18:21:11.912357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:41:17.220856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911980+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:21:11.911980+0000","last_peered":"2024-04-26T18:21:11.911980+0000","last_clean":"2024-04-26T18:21:11.911980+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:21:11.911980+0000","last_undegraded":"2024-04-26T18:21:11.911980+0000","last_fullsized":"2024-04-26T18:21:11.911980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918431+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:21:11.918431+0000","last_peered":"2024-04-26T18:21:11.918431+0000","last_clean":"2024-04-26T18:21:11.918431+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:21:11.918431+0000","last_undegraded":"2024-04-26T18:21:11.918431+0000","last_fullsized":"2024-04-26T18:21:11.918431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912150+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:21:11.912150+0000","last_peered":"2024-04-26T18:21:11.912150+0000","last_clean":"2024-04-26T18:21:11.912150+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:21:11.912150+0000","last_undegraded":"2024-04-26T18:21:11.912150+0000","last_fullsized":"2024-04-26T18:21:11.912150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901195+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:21:11.901195+0000","last_peered":"2024-04-26T18:21:11.901195+0000","last_clean":"2024-04-26T18:21:11.901195+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:21:11.901195+0000","last_undegraded":"2024-04-26T18:21:11.901195+0000","last_fullsized":"2024-04-26T18:21:11.901195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913646+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:21:11.913646+0000","last_peered":"2024-04-26T18:21:11.913646+0000","last_clean":"2024-04-26T18:21:11.913646+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:21:11.913646+0000","last_undegraded":"2024-04-26T18:21:11.913646+0000","last_fullsized":"2024-04-26T18:21:11.913646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915978+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:21:11.915978+0000","last_peered":"2024-04-26T18:21:11.915978+0000","last_clean":"2024-04-26T18:21:11.915978+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:21:11.915978+0000","last_undegraded":"2024-04-26T18:21:11.915978+0000","last_fullsized":"2024-04-26T18:21:11.915978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912417+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:21:11.912417+0000","last_peered":"2024-04-26T18:21:11.912417+0000","last_clean":"2024-04-26T18:21:11.912417+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:21:11.912417+0000","last_undegraded":"2024-04-26T18:21:11.912417+0000","last_fullsized":"2024-04-26T18:21:11.912417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914962+0000","last_change":"2024-04-26T18:21:08.910761+0000","last_active":"2024-04-26T18:21:11.914962+0000","last_peered":"2024-04-26T18:21:11.914962+0000","last_clean":"2024-04-26T18:21:11.914962+0000","last_became_active":"2024-04-26T18:21:08.908517+0000","last_became_peered":"2024-04-26T18:21:08.908517+0000","last_unstale":"2024-04-26T18:21:11.914962+0000","last_undegraded":"2024-04-26T18:21:11.914962+0000","last_fullsized":"2024-04-26T18:21:11.914962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:42:44.411236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913661+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:21:11.913661+0000","last_peered":"2024-04-26T18:21:11.913661+0000","last_clean":"2024-04-26T18:21:11.913661+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:21:11.913661+0000","last_undegraded":"2024-04-26T18:21:11.913661+0000","last_fullsized":"2024-04-26T18:21:11.913661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912671+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:21:11.912671+0000","last_peered":"2024-04-26T18:21:11.912671+0000","last_clean":"2024-04-26T18:21:11.912671+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:21:11.912671+0000","last_undegraded":"2024-04-26T18:21:11.912671+0000","last_fullsized":"2024-04-26T18:21:11.912671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901080+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:21:11.901080+0000","last_peered":"2024-04-26T18:21:11.901080+0000","last_clean":"2024-04-26T18:21:11.901080+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:21:11.901080+0000","last_undegraded":"2024-04-26T18:21:11.901080+0000","last_fullsized":"2024-04-26T18:21:11.901080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58953,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":169,"num_read_kb":147,"num_write":159,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":179135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40428,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704724,"statfs":{"total":95995035648,"available":95953637376,"internally_reserved":0,"allocated":954368,"data_stored":462787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645341,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1109220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":463023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070502,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":462943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332139,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1054873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":462412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":518295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":463683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1040384,"data_stored":518898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":462723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":462849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1054557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:21:54.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:21:54.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:54 smithi120 ceph-mon[32244]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:55.138 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:55 smithi037 ceph-mon[29171]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:55.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:55 smithi170 ceph-mon[32697]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:55 smithi120 ceph-mon[32244]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:56.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:56 smithi037 ceph-mon[29171]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:56 smithi170 ceph-mon[32697]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 -- 172.21.15.37:0/1216594462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81034c0 msgr2=0x7f55d8103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 --2- 172.21.15.37:0/1216594462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81034c0 0x7f55d8103940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f55d40099c0 tx=0x7f55d4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 -- 172.21.15.37:0/1216594462 shutdown_connections 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 --2- 172.21.15.37:0/1216594462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8103e80 0x7f55d8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 --2- 172.21.15.37:0/1216594462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d8109560 0x7f55d8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 --2- 172.21.15.37:0/1216594462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81034c0 0x7f55d8103940 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.349+0000 7f55df84e700 1 -- 172.21.15.37:0/1216594462 >> 172.21.15.37:0/1216594462 conn(0x7f55d807a5d0 msgr2=0x7f55d807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- 172.21.15.37:0/1216594462 shutdown_connections 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- 172.21.15.37:0/1216594462 wait complete. 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 Processor -- start 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- start start 2024-04-26T18:21:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d81034c0 0x7f55d8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8109560 0x7f55d80761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113710 con 0x7f55d8103e80 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113590 con 0x7f55d8109560 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.350+0000 7f55df84e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113890 con 0x7f55d81034c0 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44048/0 (socket says 172.21.15.37:44048) 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 -- 172.21.15.37:0/1384504624 learned_addr learned my addr 172.21.15.37:0/1384504624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dd5ea700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d81034c0 0x7f55d8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dddeb700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8109560 0x7f55d80761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 -- 172.21.15.37:0/1384504624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d81034c0 msgr2=0x7f55d8075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d81034c0 0x7f55d8075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 -- 172.21.15.37:0/1384504624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8109560 msgr2=0x7f55d80761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8109560 0x7f55d80761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 -- 172.21.15.37:0/1384504624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d81a80a0 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55dcde9700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009990 tx=0x7f55d40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4027070 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d4003fa0 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a8330 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4022d20 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.351+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a8780 con 0x7f55d8103e80 2024-04-26T18:21:56.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.352+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d810b9f0 con 0x7f55d8103e80 2024-04-26T18:21:56.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.354+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f55d402a070 con 0x7f55d8103e80 2024-04-26T18:21:56.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.355+0000 7f55ca7fc700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:56.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.355+0000 7f55dd5ea700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:56.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.355+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f55d40d1670 con 0x7f55d8103e80 2024-04-26T18:21:56.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.355+0000 7f55dd5ea700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 0x7f55c40a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009a10 tx=0x7f55cc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:56.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.359+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55d4031070 con 0x7f55d8103e80 2024-04-26T18:21:56.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:56 smithi120 ceph-mon[32244]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-04-26T18:21:56.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.565+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55d8063860 con 0x7f55c40a3cd0 2024-04-26T18:21:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.573+0000 7f55ca7fc700 1 -- 172.21.15.37:0/1384504624 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258708 (secure 0 0 0) 0x7f55d8063860 con 0x7f55c40a3cd0 2024-04-26T18:21:56.576 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 msgr2=0x7f55c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 0x7f55c40a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009a10 tx=0x7f55cc009190 comp rx=0 tx=0).stop 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 msgr2=0x7f55d8075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009990 tx=0x7f55d40047b0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 shutdown_connections 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f55c40a3cd0 0x7f55c40a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f55d8109560 0x7f55d80761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55d81034c0 0x7f55d8075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 --2- 172.21.15.37:0/1384504624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d8103e80 0x7f55d8075c60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 >> 172.21.15.37:0/1384504624 conn(0x7f55d807a5d0 msgr2=0x7f55d8102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 shutdown_connections 2024-04-26T18:21:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:56.576+0000 7f55df84e700 1 -- 172.21.15.37:0/1384504624 wait complete. 2024-04-26T18:21:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:21:57.261 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-26T18:21:54.682936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649555,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":310,"num_read_kb":266,"num_write":344,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383248,"kb_used_data":13296,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558576,"statfs":{"total":1151940427776,"available":1151547981824,"internally_reserved":0,"allocated":13615104,"data_stored":7496263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":63,"num_write":24,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002811"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901365+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:21:11.901365+0000","last_peered":"2024-04-26T18:21:11.901365+0000","last_clean":"2024-04-26T18:21:11.901365+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:21:11.901365+0000","last_undegraded":"2024-04-26T18:21:11.901365+0000","last_fullsized":"2024-04-26T18:21:11.901365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917530+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:21:11.917530+0000","last_peered":"2024-04-26T18:21:11.917530+0000","last_clean":"2024-04-26T18:21:11.917530+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:21:11.917530+0000","last_undegraded":"2024-04-26T18:21:11.917530+0000","last_fullsized":"2024-04-26T18:21:11.917530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917156+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:21:11.917156+0000","last_peered":"2024-04-26T18:21:11.917156+0000","last_clean":"2024-04-26T18:21:11.917156+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:21:11.917156+0000","last_undegraded":"2024-04-26T18:21:11.917156+0000","last_fullsized":"2024-04-26T18:21:11.917156+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912775+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:21:11.912775+0000","last_peered":"2024-04-26T18:21:11.912775+0000","last_clean":"2024-04-26T18:21:11.912775+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:21:11.912775+0000","last_undegraded":"2024-04-26T18:21:11.912775+0000","last_fullsized":"2024-04-26T18:21:11.912775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913458+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:21:11.913458+0000","last_peered":"2024-04-26T18:21:11.913458+0000","last_clean":"2024-04-26T18:21:11.913458+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:21:11.913458+0000","last_undegraded":"2024-04-26T18:21:11.913458+0000","last_fullsized":"2024-04-26T18:21:11.913458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900548+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:21:11.900548+0000","last_peered":"2024-04-26T18:21:11.900548+0000","last_clean":"2024-04-26T18:21:11.900548+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:21:11.900548+0000","last_undegraded":"2024-04-26T18:21:11.900548+0000","last_fullsized":"2024-04-26T18:21:11.900548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911559+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:21:11.911559+0000","last_peered":"2024-04-26T18:21:11.911559+0000","last_clean":"2024-04-26T18:21:11.911559+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:21:11.911559+0000","last_undegraded":"2024-04-26T18:21:11.911559+0000","last_fullsized":"2024-04-26T18:21:11.911559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914698+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:21:11.914698+0000","last_peered":"2024-04-26T18:21:11.914698+0000","last_clean":"2024-04-26T18:21:11.914698+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:21:11.914698+0000","last_undegraded":"2024-04-26T18:21:11.914698+0000","last_fullsized":"2024-04-26T18:21:11.914698+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914292+0000","last_change":"2024-04-26T18:21:08.902438+0000","last_active":"2024-04-26T18:21:11.914292+0000","last_peered":"2024-04-26T18:21:11.914292+0000","last_clean":"2024-04-26T18:21:11.914292+0000","last_became_active":"2024-04-26T18:21:08.902094+0000","last_became_peered":"2024-04-26T18:21:08.902094+0000","last_unstale":"2024-04-26T18:21:11.914292+0000","last_undegraded":"2024-04-26T18:21:11.914292+0000","last_fullsized":"2024-04-26T18:21:11.914292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:03.731129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900947+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:21:11.900947+0000","last_peered":"2024-04-26T18:21:11.900947+0000","last_clean":"2024-04-26T18:21:11.900947+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:21:11.900947+0000","last_undegraded":"2024-04-26T18:21:11.900947+0000","last_fullsized":"2024-04-26T18:21:11.900947+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913122+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:21:11.913122+0000","last_peered":"2024-04-26T18:21:11.913122+0000","last_clean":"2024-04-26T18:21:11.913122+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:21:11.913122+0000","last_undegraded":"2024-04-26T18:21:11.913122+0000","last_fullsized":"2024-04-26T18:21:11.913122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912635+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:21:11.912635+0000","last_peered":"2024-04-26T18:21:11.912635+0000","last_clean":"2024-04-26T18:21:11.912635+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:21:11.912635+0000","last_undegraded":"2024-04-26T18:21:11.912635+0000","last_fullsized":"2024-04-26T18:21:11.912635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912618+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:21:11.912618+0000","last_peered":"2024-04-26T18:21:11.912618+0000","last_clean":"2024-04-26T18:21:11.912618+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:21:11.912618+0000","last_undegraded":"2024-04-26T18:21:11.912618+0000","last_fullsized":"2024-04-26T18:21:11.912618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916940+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:21:11.916940+0000","last_peered":"2024-04-26T18:21:11.916940+0000","last_clean":"2024-04-26T18:21:11.916940+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:21:11.916940+0000","last_undegraded":"2024-04-26T18:21:11.916940+0000","last_fullsized":"2024-04-26T18:21:11.916940+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916708+0000","last_change":"2024-04-26T18:21:08.912832+0000","last_active":"2024-04-26T18:21:11.916708+0000","last_peered":"2024-04-26T18:21:11.916708+0000","last_clean":"2024-04-26T18:21:11.916708+0000","last_became_active":"2024-04-26T18:21:08.912269+0000","last_became_peered":"2024-04-26T18:21:08.912269+0000","last_unstale":"2024-04-26T18:21:11.916708+0000","last_undegraded":"2024-04-26T18:21:11.916708+0000","last_fullsized":"2024-04-26T18:21:11.916708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:37:15.142761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913470+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:21:11.913470+0000","last_peered":"2024-04-26T18:21:11.913470+0000","last_clean":"2024-04-26T18:21:11.913470+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:21:11.913470+0000","last_undegraded":"2024-04-26T18:21:11.913470+0000","last_fullsized":"2024-04-26T18:21:11.913470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901484+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:21:11.901484+0000","last_peered":"2024-04-26T18:21:11.901484+0000","last_clean":"2024-04-26T18:21:11.901484+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:21:11.901484+0000","last_undegraded":"2024-04-26T18:21:11.901484+0000","last_fullsized":"2024-04-26T18:21:11.901484+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901347+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:21:11.901347+0000","last_peered":"2024-04-26T18:21:11.901347+0000","last_clean":"2024-04-26T18:21:11.901347+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:21:11.901347+0000","last_undegraded":"2024-04-26T18:21:11.901347+0000","last_fullsized":"2024-04-26T18:21:11.901347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914125+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:21:11.914125+0000","last_peered":"2024-04-26T18:21:11.914125+0000","last_clean":"2024-04-26T18:21:11.914125+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:21:11.914125+0000","last_undegraded":"2024-04-26T18:21:11.914125+0000","last_fullsized":"2024-04-26T18:21:11.914125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950922+0000","last_change":"2024-04-26T18:21:05.874286+0000","last_active":"2024-04-26T18:21:24.950922+0000","last_peered":"2024-04-26T18:21:24.950922+0000","last_clean":"2024-04-26T18:21:24.950922+0000","last_became_active":"2024-04-26T18:21:05.873882+0000","last_became_peered":"2024-04-26T18:21:05.873882+0000","last_unstale":"2024-04-26T18:21:24.950922+0000","last_undegraded":"2024-04-26T18:21:24.950922+0000","last_fullsized":"2024-04-26T18:21:24.950922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:26.225532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915966+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:21:11.915966+0000","last_peered":"2024-04-26T18:21:11.915966+0000","last_clean":"2024-04-26T18:21:11.915966+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:21:11.915966+0000","last_undegraded":"2024-04-26T18:21:11.915966+0000","last_fullsized":"2024-04-26T18:21:11.915966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918652+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:21:11.918652+0000","last_peered":"2024-04-26T18:21:11.918652+0000","last_clean":"2024-04-26T18:21:11.918652+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:21:11.918652+0000","last_undegraded":"2024-04-26T18:21:11.918652+0000","last_fullsized":"2024-04-26T18:21:11.918652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917725+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:21:11.917725+0000","last_peered":"2024-04-26T18:21:11.917725+0000","last_clean":"2024-04-26T18:21:11.917725+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:21:11.917725+0000","last_undegraded":"2024-04-26T18:21:11.917725+0000","last_fullsized":"2024-04-26T18:21:11.917725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950231+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:21:24.950231+0000","last_peered":"2024-04-26T18:21:24.950231+0000","last_clean":"2024-04-26T18:21:24.950231+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:21:24.950231+0000","last_undegraded":"2024-04-26T18:21:24.950231+0000","last_fullsized":"2024-04-26T18:21:24.950231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913425+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:21:11.913425+0000","last_peered":"2024-04-26T18:21:11.913425+0000","last_clean":"2024-04-26T18:21:11.913425+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:21:11.913425+0000","last_undegraded":"2024-04-26T18:21:11.913425+0000","last_fullsized":"2024-04-26T18:21:11.913425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.919441+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:21:11.919441+0000","last_peered":"2024-04-26T18:21:11.919441+0000","last_clean":"2024-04-26T18:21:11.919441+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:21:11.919441+0000","last_undegraded":"2024-04-26T18:21:11.919441+0000","last_fullsized":"2024-04-26T18:21:11.919441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913206+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:21:11.913206+0000","last_peered":"2024-04-26T18:21:11.913206+0000","last_clean":"2024-04-26T18:21:11.913206+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:21:11.913206+0000","last_undegraded":"2024-04-26T18:21:11.913206+0000","last_fullsized":"2024-04-26T18:21:11.913206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:52.995438+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:21:52.995438+0000","last_peered":"2024-04-26T18:21:52.995438+0000","last_clean":"2024-04-26T18:21:52.995438+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:21:52.995438+0000","last_undegraded":"2024-04-26T18:21:52.995438+0000","last_fullsized":"2024-04-26T18:21:52.995438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915281+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:21:11.915281+0000","last_peered":"2024-04-26T18:21:11.915281+0000","last_clean":"2024-04-26T18:21:11.915281+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:21:11.915281+0000","last_undegraded":"2024-04-26T18:21:11.915281+0000","last_fullsized":"2024-04-26T18:21:11.915281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.948964+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:21:24.948964+0000","last_peered":"2024-04-26T18:21:24.948964+0000","last_clean":"2024-04-26T18:21:24.948964+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:21:24.948964+0000","last_undegraded":"2024-04-26T18:21:24.948964+0000","last_fullsized":"2024-04-26T18:21:24.948964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912954+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:21:11.912954+0000","last_peered":"2024-04-26T18:21:11.912954+0000","last_clean":"2024-04-26T18:21:11.912954+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:21:11.912954+0000","last_undegraded":"2024-04-26T18:21:11.912954+0000","last_fullsized":"2024-04-26T18:21:11.912954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949265+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:21:24.949265+0000","last_peered":"2024-04-26T18:21:24.949265+0000","last_clean":"2024-04-26T18:21:24.949265+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:21:24.949265+0000","last_undegraded":"2024-04-26T18:21:24.949265+0000","last_fullsized":"2024-04-26T18:21:24.949265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914789+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:21:11.914789+0000","last_peered":"2024-04-26T18:21:11.914789+0000","last_clean":"2024-04-26T18:21:11.914789+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:21:11.914789+0000","last_undegraded":"2024-04-26T18:21:11.914789+0000","last_fullsized":"2024-04-26T18:21:11.914789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913595+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:21:11.913595+0000","last_peered":"2024-04-26T18:21:11.913595+0000","last_clean":"2024-04-26T18:21:11.913595+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:21:11.913595+0000","last_undegraded":"2024-04-26T18:21:11.913595+0000","last_fullsized":"2024-04-26T18:21:11.913595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901023+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:21:11.901023+0000","last_peered":"2024-04-26T18:21:11.901023+0000","last_clean":"2024-04-26T18:21:11.901023+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:21:11.901023+0000","last_undegraded":"2024-04-26T18:21:11.901023+0000","last_fullsized":"2024-04-26T18:21:11.901023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.988880+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:21:48.988880+0000","last_peered":"2024-04-26T18:21:48.988880+0000","last_clean":"2024-04-26T18:21:48.988880+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:21:48.988880+0000","last_undegraded":"2024-04-26T18:21:48.988880+0000","last_fullsized":"2024-04-26T18:21:48.988880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914004+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:21:11.914004+0000","last_peered":"2024-04-26T18:21:11.914004+0000","last_clean":"2024-04-26T18:21:11.914004+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:21:11.914004+0000","last_undegraded":"2024-04-26T18:21:11.914004+0000","last_fullsized":"2024-04-26T18:21:11.914004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941023+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:21:23.941023+0000","last_peered":"2024-04-26T18:21:23.941023+0000","last_clean":"2024-04-26T18:21:23.941023+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:21:23.941023+0000","last_undegraded":"2024-04-26T18:21:23.941023+0000","last_fullsized":"2024-04-26T18:21:23.941023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917530+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:21:11.917530+0000","last_peered":"2024-04-26T18:21:11.917530+0000","last_clean":"2024-04-26T18:21:11.917530+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:21:11.917530+0000","last_undegraded":"2024-04-26T18:21:11.917530+0000","last_fullsized":"2024-04-26T18:21:11.917530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.942257+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:21:23.942257+0000","last_peered":"2024-04-26T18:21:23.942257+0000","last_clean":"2024-04-26T18:21:23.942257+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:21:23.942257+0000","last_undegraded":"2024-04-26T18:21:23.942257+0000","last_fullsized":"2024-04-26T18:21:23.942257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913605+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:21:11.913605+0000","last_peered":"2024-04-26T18:21:11.913605+0000","last_clean":"2024-04-26T18:21:11.913605+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:21:11.913605+0000","last_undegraded":"2024-04-26T18:21:11.913605+0000","last_fullsized":"2024-04-26T18:21:11.913605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'25","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:49.991397+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:21:49.991397+0000","last_peered":"2024-04-26T18:21:49.991397+0000","last_clean":"2024-04-26T18:21:49.991397+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:21:49.991397+0000","last_undegraded":"2024-04-26T18:21:49.991397+0000","last_fullsized":"2024-04-26T18:21:49.991397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":55,"num_read_kb":80,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912144+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:21:11.912144+0000","last_peered":"2024-04-26T18:21:11.912144+0000","last_clean":"2024-04-26T18:21:11.912144+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:21:11.912144+0000","last_undegraded":"2024-04-26T18:21:11.912144+0000","last_fullsized":"2024-04-26T18:21:11.912144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949570+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:21:24.949570+0000","last_peered":"2024-04-26T18:21:24.949570+0000","last_clean":"2024-04-26T18:21:24.949570+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:21:24.949570+0000","last_undegraded":"2024-04-26T18:21:24.949570+0000","last_fullsized":"2024-04-26T18:21:24.949570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912116+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:21:11.912116+0000","last_peered":"2024-04-26T18:21:11.912116+0000","last_clean":"2024-04-26T18:21:11.912116+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:21:11.912116+0000","last_undegraded":"2024-04-26T18:21:11.912116+0000","last_fullsized":"2024-04-26T18:21:11.912116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941380+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:21:23.941380+0000","last_peered":"2024-04-26T18:21:23.941380+0000","last_clean":"2024-04-26T18:21:23.941380+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:21:23.941380+0000","last_undegraded":"2024-04-26T18:21:23.941380+0000","last_fullsized":"2024-04-26T18:21:23.941380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913091+0000","last_change":"2024-04-26T18:21:08.911668+0000","last_active":"2024-04-26T18:21:11.913091+0000","last_peered":"2024-04-26T18:21:11.913091+0000","last_clean":"2024-04-26T18:21:11.913091+0000","last_became_active":"2024-04-26T18:21:08.911321+0000","last_became_peered":"2024-04-26T18:21:08.911321+0000","last_unstale":"2024-04-26T18:21:11.913091+0000","last_undegraded":"2024-04-26T18:21:11.913091+0000","last_fullsized":"2024-04-26T18:21:11.913091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.457356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.989810+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:21:48.989810+0000","last_peered":"2024-04-26T18:21:48.989810+0000","last_clean":"2024-04-26T18:21:48.989810+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:21:48.989810+0000","last_undegraded":"2024-04-26T18:21:48.989810+0000","last_fullsized":"2024-04-26T18:21:48.989810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911387+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:21:11.911387+0000","last_peered":"2024-04-26T18:21:11.911387+0000","last_clean":"2024-04-26T18:21:11.911387+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:21:11.911387+0000","last_undegraded":"2024-04-26T18:21:11.911387+0000","last_fullsized":"2024-04-26T18:21:11.911387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.950989+0000","last_change":"2024-04-26T18:21:06.144981+0000","last_active":"2024-04-26T18:21:24.950989+0000","last_peered":"2024-04-26T18:21:24.950989+0000","last_clean":"2024-04-26T18:21:24.950989+0000","last_became_active":"2024-04-26T18:21:06.144573+0000","last_became_peered":"2024-04-26T18:21:06.144573+0000","last_unstale":"2024-04-26T18:21:24.950989+0000","last_undegraded":"2024-04-26T18:21:24.950989+0000","last_fullsized":"2024-04-26T18:21:24.950989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:58:43.182801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912078+0000","last_change":"2024-04-26T18:21:08.899461+0000","last_active":"2024-04-26T18:21:11.912078+0000","last_peered":"2024-04-26T18:21:11.912078+0000","last_clean":"2024-04-26T18:21:11.912078+0000","last_became_active":"2024-04-26T18:21:08.899044+0000","last_became_peered":"2024-04-26T18:21:08.899044+0000","last_unstale":"2024-04-26T18:21:11.912078+0000","last_undegraded":"2024-04-26T18:21:11.912078+0000","last_fullsized":"2024-04-26T18:21:11.912078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:20.427553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:25.946239+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:21:25.946239+0000","last_peered":"2024-04-26T18:21:25.946239+0000","last_clean":"2024-04-26T18:21:25.946239+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:21:25.946239+0000","last_undegraded":"2024-04-26T18:21:25.946239+0000","last_fullsized":"2024-04-26T18:21:25.946239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901349+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:21:11.901349+0000","last_peered":"2024-04-26T18:21:11.901349+0000","last_clean":"2024-04-26T18:21:11.901349+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:21:11.901349+0000","last_undegraded":"2024-04-26T18:21:11.901349+0000","last_fullsized":"2024-04-26T18:21:11.901349+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901696+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:21:11.901696+0000","last_peered":"2024-04-26T18:21:11.901696+0000","last_clean":"2024-04-26T18:21:11.901696+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:21:11.901696+0000","last_undegraded":"2024-04-26T18:21:11.901696+0000","last_fullsized":"2024-04-26T18:21:11.901696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900768+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:21:11.900768+0000","last_peered":"2024-04-26T18:21:11.900768+0000","last_clean":"2024-04-26T18:21:11.900768+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:21:11.900768+0000","last_undegraded":"2024-04-26T18:21:11.900768+0000","last_fullsized":"2024-04-26T18:21:11.900768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911960+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:21:11.911960+0000","last_peered":"2024-04-26T18:21:11.911960+0000","last_clean":"2024-04-26T18:21:11.911960+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:21:11.911960+0000","last_undegraded":"2024-04-26T18:21:11.911960+0000","last_fullsized":"2024-04-26T18:21:11.911960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900592+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:21:11.900592+0000","last_peered":"2024-04-26T18:21:11.900592+0000","last_clean":"2024-04-26T18:21:11.900592+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:21:11.900592+0000","last_undegraded":"2024-04-26T18:21:11.900592+0000","last_fullsized":"2024-04-26T18:21:11.900592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912720+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:21:11.912720+0000","last_peered":"2024-04-26T18:21:11.912720+0000","last_clean":"2024-04-26T18:21:11.912720+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:21:11.912720+0000","last_undegraded":"2024-04-26T18:21:11.912720+0000","last_fullsized":"2024-04-26T18:21:11.912720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:48.989215+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:21:48.989215+0000","last_peered":"2024-04-26T18:21:48.989215+0000","last_clean":"2024-04-26T18:21:48.989215+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:21:48.989215+0000","last_undegraded":"2024-04-26T18:21:48.989215+0000","last_fullsized":"2024-04-26T18:21:48.989215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915311+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:21:11.915311+0000","last_peered":"2024-04-26T18:21:11.915311+0000","last_clean":"2024-04-26T18:21:11.915311+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:21:11.915311+0000","last_undegraded":"2024-04-26T18:21:11.915311+0000","last_fullsized":"2024-04-26T18:21:11.915311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:26.954972+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:21:26.954972+0000","last_peered":"2024-04-26T18:21:26.954972+0000","last_clean":"2024-04-26T18:21:26.954972+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:21:26.954972+0000","last_undegraded":"2024-04-26T18:21:26.954972+0000","last_fullsized":"2024-04-26T18:21:26.954972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911616+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:21:11.911616+0000","last_peered":"2024-04-26T18:21:11.911616+0000","last_clean":"2024-04-26T18:21:11.911616+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:21:11.911616+0000","last_undegraded":"2024-04-26T18:21:11.911616+0000","last_fullsized":"2024-04-26T18:21:11.911616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912234+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:21:11.912234+0000","last_peered":"2024-04-26T18:21:11.912234+0000","last_clean":"2024-04-26T18:21:11.912234+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:21:11.912234+0000","last_undegraded":"2024-04-26T18:21:11.912234+0000","last_fullsized":"2024-04-26T18:21:11.912234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.949404+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:21:24.949404+0000","last_peered":"2024-04-26T18:21:24.949404+0000","last_clean":"2024-04-26T18:21:24.949404+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:21:24.949404+0000","last_undegraded":"2024-04-26T18:21:24.949404+0000","last_fullsized":"2024-04-26T18:21:24.949404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.939822+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:21:23.939822+0000","last_peered":"2024-04-26T18:21:23.939822+0000","last_clean":"2024-04-26T18:21:23.939822+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:21:23.939822+0000","last_undegraded":"2024-04-26T18:21:23.939822+0000","last_fullsized":"2024-04-26T18:21:23.939822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912127+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:21:11.912127+0000","last_peered":"2024-04-26T18:21:11.912127+0000","last_clean":"2024-04-26T18:21:11.912127+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:21:11.912127+0000","last_undegraded":"2024-04-26T18:21:11.912127+0000","last_fullsized":"2024-04-26T18:21:11.912127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:47.916226+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:21:47.916226+0000","last_peered":"2024-04-26T18:21:47.916226+0000","last_clean":"2024-04-26T18:21:47.916226+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:21:47.916226+0000","last_undegraded":"2024-04-26T18:21:47.916226+0000","last_fullsized":"2024-04-26T18:21:47.916226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.916155+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:21:11.916155+0000","last_peered":"2024-04-26T18:21:11.916155+0000","last_clean":"2024-04-26T18:21:11.916155+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:21:11.916155+0000","last_undegraded":"2024-04-26T18:21:11.916155+0000","last_fullsized":"2024-04-26T18:21:11.916155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.940910+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:21:23.940910+0000","last_peered":"2024-04-26T18:21:23.940910+0000","last_clean":"2024-04-26T18:21:23.940910+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:21:23.940910+0000","last_undegraded":"2024-04-26T18:21:23.940910+0000","last_fullsized":"2024-04-26T18:21:23.940910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917651+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:21:11.917651+0000","last_peered":"2024-04-26T18:21:11.917651+0000","last_clean":"2024-04-26T18:21:11.917651+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:21:11.917651+0000","last_undegraded":"2024-04-26T18:21:11.917651+0000","last_fullsized":"2024-04-26T18:21:11.917651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.939922+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:21:23.939922+0000","last_peered":"2024-04-26T18:21:23.939922+0000","last_clean":"2024-04-26T18:21:23.939922+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:21:23.939922+0000","last_undegraded":"2024-04-26T18:21:23.939922+0000","last_fullsized":"2024-04-26T18:21:23.939922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911948+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:21:11.911948+0000","last_peered":"2024-04-26T18:21:11.911948+0000","last_clean":"2024-04-26T18:21:11.911948+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:21:11.911948+0000","last_undegraded":"2024-04-26T18:21:11.911948+0000","last_fullsized":"2024-04-26T18:21:11.911948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941723+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:21:23.941723+0000","last_peered":"2024-04-26T18:21:23.941723+0000","last_clean":"2024-04-26T18:21:23.941723+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:21:23.941723+0000","last_undegraded":"2024-04-26T18:21:23.941723+0000","last_fullsized":"2024-04-26T18:21:23.941723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.900970+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:21:11.900970+0000","last_peered":"2024-04-26T18:21:11.900970+0000","last_clean":"2024-04-26T18:21:11.900970+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:21:11.900970+0000","last_undegraded":"2024-04-26T18:21:11.900970+0000","last_fullsized":"2024-04-26T18:21:11.900970+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912237+0000","last_change":"2024-04-26T18:21:05.875607+0000","last_active":"2024-04-26T18:21:11.912237+0000","last_peered":"2024-04-26T18:21:11.912237+0000","last_clean":"2024-04-26T18:21:11.912237+0000","last_became_active":"2024-04-26T18:21:05.875255+0000","last_became_peered":"2024-04-26T18:21:05.875255+0000","last_unstale":"2024-04-26T18:21:11.912237+0000","last_undegraded":"2024-04-26T18:21:11.912237+0000","last_fullsized":"2024-04-26T18:21:11.912237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:47:28.674657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915267+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:21:11.915267+0000","last_peered":"2024-04-26T18:21:11.915267+0000","last_clean":"2024-04-26T18:21:11.915267+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:21:11.915267+0000","last_undegraded":"2024-04-26T18:21:11.915267+0000","last_fullsized":"2024-04-26T18:21:11.915267+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:23.941508+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:21:23.941508+0000","last_peered":"2024-04-26T18:21:23.941508+0000","last_clean":"2024-04-26T18:21:23.941508+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:21:23.941508+0000","last_undegraded":"2024-04-26T18:21:23.941508+0000","last_fullsized":"2024-04-26T18:21:23.941508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912076+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:21:11.912076+0000","last_peered":"2024-04-26T18:21:11.912076+0000","last_clean":"2024-04-26T18:21:11.912076+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:21:11.912076+0000","last_undegraded":"2024-04-26T18:21:11.912076+0000","last_fullsized":"2024-04-26T18:21:11.912076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:24.951329+0000","last_change":"2024-04-26T18:21:06.146648+0000","last_active":"2024-04-26T18:21:24.951329+0000","last_peered":"2024-04-26T18:21:24.951329+0000","last_clean":"2024-04-26T18:21:24.951329+0000","last_became_active":"2024-04-26T18:21:06.146016+0000","last_became_peered":"2024-04-26T18:21:06.146016+0000","last_unstale":"2024-04-26T18:21:24.951329+0000","last_undegraded":"2024-04-26T18:21:24.951329+0000","last_fullsized":"2024-04-26T18:21:24.951329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:52.796019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913404+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:21:11.913404+0000","last_peered":"2024-04-26T18:21:11.913404+0000","last_clean":"2024-04-26T18:21:11.913404+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:21:11.913404+0000","last_undegraded":"2024-04-26T18:21:11.913404+0000","last_fullsized":"2024-04-26T18:21:11.913404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918076+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:21:11.918076+0000","last_peered":"2024-04-26T18:21:11.918076+0000","last_clean":"2024-04-26T18:21:11.918076+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:21:11.918076+0000","last_undegraded":"2024-04-26T18:21:11.918076+0000","last_fullsized":"2024-04-26T18:21:11.918076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912466+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:21:11.912466+0000","last_peered":"2024-04-26T18:21:11.912466+0000","last_clean":"2024-04-26T18:21:11.912466+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:21:11.912466+0000","last_undegraded":"2024-04-26T18:21:11.912466+0000","last_fullsized":"2024-04-26T18:21:11.912466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.917270+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:21:11.917270+0000","last_peered":"2024-04-26T18:21:11.917270+0000","last_clean":"2024-04-26T18:21:11.917270+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:21:11.917270+0000","last_undegraded":"2024-04-26T18:21:11.917270+0000","last_fullsized":"2024-04-26T18:21:11.917270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912825+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:21:11.912825+0000","last_peered":"2024-04-26T18:21:11.912825+0000","last_clean":"2024-04-26T18:21:11.912825+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:21:11.912825+0000","last_undegraded":"2024-04-26T18:21:11.912825+0000","last_fullsized":"2024-04-26T18:21:11.912825+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901199+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:21:11.901199+0000","last_peered":"2024-04-26T18:21:11.901199+0000","last_clean":"2024-04-26T18:21:11.901199+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:21:11.901199+0000","last_undegraded":"2024-04-26T18:21:11.901199+0000","last_fullsized":"2024-04-26T18:21:11.901199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912357+0000","last_change":"2024-04-26T18:21:08.911306+0000","last_active":"2024-04-26T18:21:11.912357+0000","last_peered":"2024-04-26T18:21:11.912357+0000","last_clean":"2024-04-26T18:21:11.912357+0000","last_became_active":"2024-04-26T18:21:08.910942+0000","last_became_peered":"2024-04-26T18:21:08.910942+0000","last_unstale":"2024-04-26T18:21:11.912357+0000","last_undegraded":"2024-04-26T18:21:11.912357+0000","last_fullsized":"2024-04-26T18:21:11.912357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:41:17.220856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.911980+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:21:11.911980+0000","last_peered":"2024-04-26T18:21:11.911980+0000","last_clean":"2024-04-26T18:21:11.911980+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:21:11.911980+0000","last_undegraded":"2024-04-26T18:21:11.911980+0000","last_fullsized":"2024-04-26T18:21:11.911980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.918431+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:21:11.918431+0000","last_peered":"2024-04-26T18:21:11.918431+0000","last_clean":"2024-04-26T18:21:11.918431+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:21:11.918431+0000","last_undegraded":"2024-04-26T18:21:11.918431+0000","last_fullsized":"2024-04-26T18:21:11.918431+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912150+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:21:11.912150+0000","last_peered":"2024-04-26T18:21:11.912150+0000","last_clean":"2024-04-26T18:21:11.912150+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:21:11.912150+0000","last_undegraded":"2024-04-26T18:21:11.912150+0000","last_fullsized":"2024-04-26T18:21:11.912150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901195+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:21:11.901195+0000","last_peered":"2024-04-26T18:21:11.901195+0000","last_clean":"2024-04-26T18:21:11.901195+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:21:11.901195+0000","last_undegraded":"2024-04-26T18:21:11.901195+0000","last_fullsized":"2024-04-26T18:21:11.901195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913646+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:21:11.913646+0000","last_peered":"2024-04-26T18:21:11.913646+0000","last_clean":"2024-04-26T18:21:11.913646+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:21:11.913646+0000","last_undegraded":"2024-04-26T18:21:11.913646+0000","last_fullsized":"2024-04-26T18:21:11.913646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.915978+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:21:11.915978+0000","last_peered":"2024-04-26T18:21:11.915978+0000","last_clean":"2024-04-26T18:21:11.915978+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:21:11.915978+0000","last_undegraded":"2024-04-26T18:21:11.915978+0000","last_fullsized":"2024-04-26T18:21:11.915978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912417+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:21:11.912417+0000","last_peered":"2024-04-26T18:21:11.912417+0000","last_clean":"2024-04-26T18:21:11.912417+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:21:11.912417+0000","last_undegraded":"2024-04-26T18:21:11.912417+0000","last_fullsized":"2024-04-26T18:21:11.912417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.914962+0000","last_change":"2024-04-26T18:21:08.910761+0000","last_active":"2024-04-26T18:21:11.914962+0000","last_peered":"2024-04-26T18:21:11.914962+0000","last_clean":"2024-04-26T18:21:11.914962+0000","last_became_active":"2024-04-26T18:21:08.908517+0000","last_became_peered":"2024-04-26T18:21:08.908517+0000","last_unstale":"2024-04-26T18:21:11.914962+0000","last_undegraded":"2024-04-26T18:21:11.914962+0000","last_fullsized":"2024-04-26T18:21:11.914962+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:42:44.411236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.913661+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:21:11.913661+0000","last_peered":"2024-04-26T18:21:11.913661+0000","last_clean":"2024-04-26T18:21:11.913661+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:21:11.913661+0000","last_undegraded":"2024-04-26T18:21:11.913661+0000","last_fullsized":"2024-04-26T18:21:11.913661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.912671+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:21:11.912671+0000","last_peered":"2024-04-26T18:21:11.912671+0000","last_clean":"2024-04-26T18:21:11.912671+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:21:11.912671+0000","last_undegraded":"2024-04-26T18:21:11.912671+0000","last_fullsized":"2024-04-26T18:21:11.912671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-26T18:21:11.901080+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:21:11.901080+0000","last_peered":"2024-04-26T18:21:11.901080+0000","last_clean":"2024-04-26T18:21:11.901080+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:21:11.901080+0000","last_undegraded":"2024-04-26T18:21:11.901080+0000","last_fullsized":"2024-04-26T18:21:11.901080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58953,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":174,"num_read_kb":147,"num_write":161,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":179135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40428,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704724,"statfs":{"total":95995035648,"available":95953637376,"internally_reserved":0,"allocated":954368,"data_stored":462787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645341,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1109220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":463023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070503,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":462943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332140,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1054873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":462412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":518295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":463683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40512,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704640,"statfs":{"total":95995035648,"available":95953551360,"internally_reserved":0,"allocated":1040384,"data_stored":518898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":462723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":462849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1054557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:21:57.265 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:21:57.265 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T18:21:57.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T18:21:57.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T18:21:57.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T18:21:57.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-26T18:21:57.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:21:57.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-26T18:21:57.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-26T18:21:57.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:21:57.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-26T18:21:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T18:21:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:21:57.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:21:57.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-26T18:21:57.514 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-26T18:21:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:58 smithi037 ceph-mon[29171]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:21:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-26T18:21:58.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:58 smithi170 ceph-mon[32697]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:21:58.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: pgmap v318: 97 pgs: 97 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-26T18:21:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:58 smithi120 ceph-mon[32244]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 -- 172.21.15.37:0/3738219512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 msgr2=0x7efca81027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 --2- 172.21.15.37:0/3738219512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81027c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efc980099c0 tx=0x7efc98009cd0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 -- 172.21.15.37:0/3738219512 shutdown_connections 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 --2- 172.21.15.37:0/3738219512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 0x7efca810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 --2- 172.21.15.37:0/3738219512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81027c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 --2- 172.21.15.37:0/3738219512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 0x7efca8103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.165+0000 7efcae608700 1 -- 172.21.15.37:0/3738219512 >> 172.21.15.37:0/3738219512 conn(0x7efca80fd940 msgr2=0x7efca80ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:59.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.166+0000 7efcae608700 1 -- 172.21.15.37:0/3738219512 shutdown_connections 2024-04-26T18:21:59.167 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:21:59 smithi037 ceph-mon[29171]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:21:59.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.166+0000 7efcae608700 1 -- 172.21.15.37:0/3738219512 wait complete. 2024-04-26T18:21:59.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.166+0000 7efcae608700 1 Processor -- start 2024-04-26T18:21:59.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.166+0000 7efcae608700 1 -- start start 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 0x7efca8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 0x7efca81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efca810de90 con 0x7efca81023a0 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efca810dd10 con 0x7efca81035a0 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcae608700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efca810e010 con 0x7efca8103f40 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44066/0 (socket says 172.21.15.37:44066) 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 -- 172.21.15.37:0/3149156022 learned_addr learned my addr 172.21.15.37:0/3149156022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca77fe700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 0x7efca8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efcacba5700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 0x7efca81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 -- 172.21.15.37:0/3149156022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 msgr2=0x7efca81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 0x7efca81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 -- 172.21.15.37:0/3149156022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 msgr2=0x7efca8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 0x7efca8199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 -- 172.21.15.37:0/3149156022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca81a1250 con 0x7efca81023a0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.167+0000 7efca7fff700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc98009f50 tx=0x7efc980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.168+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98027070 con 0x7efca81023a0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.168+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc980041a0 con 0x7efca81023a0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.168+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81a14e0 con 0x7efca81023a0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.168+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a19c0 con 0x7efca81023a0 2024-04-26T18:21:59.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.168+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800f460 con 0x7efca81023a0 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.169+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8051960 con 0x7efca81023a0 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.170+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc9802a070 con 0x7efca81023a0 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.170+0000 7efca57fa700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 0x7efc900a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.171+0000 7efca77fe700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 0x7efc900a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.171+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7efc980d1570 con 0x7efca81023a0 2024-04-26T18:21:59.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.171+0000 7efca77fe700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 0x7efc900a6240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006fd0 tx=0x7efc9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:59.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.175+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc9805eff0 con 0x7efca81023a0 2024-04-26T18:21:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:21:59 smithi170 ceph-mon[32697]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:21:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.466+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7efca81a1de0 con 0x7efca81023a0 2024-04-26T18:21:59.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.469+0000 7efca57fa700 1 -- 172.21.15.37:0/3149156022 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7efc98013070 con 0x7efca81023a0 2024-04-26T18:21:59.469 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-26T18:21:59.469 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQDH8CtmLZTnGxAAG6dBfit/KEfFt/kFa5vHtw== 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 msgr2=0x7efc900a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 0x7efc900a6240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006fd0 tx=0x7efc9c008040 comp rx=0 tx=0).stop 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 msgr2=0x7efca81995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc98009f50 tx=0x7efc980047b0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 shutdown_connections 2024-04-26T18:21:59.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efc900a3d80 0x7efc900a6240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efca8103f40 0x7efca81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efca81023a0 0x7efca81995f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 --2- 172.21.15.37:0/3149156022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efca81035a0 0x7efca8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 >> 172.21.15.37:0/3149156022 conn(0x7efca80fd940 msgr2=0x7efca80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 shutdown_connections 2024-04-26T18:21:59.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:21:59.471+0000 7efcae608700 1 -- 172.21.15.37:0/3149156022 wait complete. 2024-04-26T18:21:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:21:59 smithi120 ceph-mon[32244]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:22:00.078 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:22:00.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T18:22:00.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T18:22:00.156 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T18:22:00.156 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T18:22:00.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T18:22:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:00 smithi170 ceph-mon[32697]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-26T18:22:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:00 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:22:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:00 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:22:00.449 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:00 smithi037 ceph-mon[29171]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-26T18:22:00.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:00 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:22:00.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:00 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:22:00.530 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:00 smithi120 ceph-mon[32244]: pgmap v321: 97 pgs: 1 peering, 96 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 4 op/s; 13 B/s, 0 objects/s recovering 2024-04-26T18:22:00.530 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:00 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:22:00.530 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:00 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3149156022' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:22:02.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:02 smithi037 ceph-mon[29171]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 -- 172.21.15.37:0/1608643388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea0109490 msgr2=0x7f5ea0109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 --2- 172.21.15.37:0/1608643388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea0109490 0x7f5ea0109870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0099c0 tx=0x7f5e9c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 -- 172.21.15.37:0/1608643388 shutdown_connections 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 --2- 172.21.15.37:0/1608643388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006c580 0x7f5ea01102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 --2- 172.21.15.37:0/1608643388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea0109490 0x7f5ea0109870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 --2- 172.21.15.37:0/1608643388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea006bbc0 0x7f5ea006c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 -- 172.21.15.37:0/1608643388 >> 172.21.15.37:0/1608643388 conn(0x7f5ea0078390 msgr2=0x7f5ea00787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 -- 172.21.15.37:0/1608643388 shutdown_connections 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.193+0000 7f5ea875d700 1 -- 172.21.15.37:0/1608643388 wait complete. 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 Processor -- start 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 -- start start 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 0x7f5ea019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea0109490 0x7f5ea0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0113610 con 0x7f5ea006c580 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0113490 con 0x7f5ea006bbc0 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea875d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0113790 con 0x7f5ea0109490 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea5cf8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea5cf8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44086/0 (socket says 172.21.15.37:44086) 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.194+0000 7f5ea5cf8700 1 -- 172.21.15.37:0/3717389822 learned_addr learned my addr 172.21.15.37:0/3717389822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea6cfa700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea0109490 0x7f5ea0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea64f9700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 0x7f5ea019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 -- 172.21.15.37:0/3717389822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea0109490 msgr2=0x7f5ea0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea0109490 0x7f5ea0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 -- 172.21.15.37:0/3717389822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 msgr2=0x7f5ea019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 0x7f5ea019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 -- 172.21.15.37:0/3717389822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea019a0f0 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea64f9700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 0x7f5ea019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea5cf8700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bf20 tx=0x7f5e9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90012b70 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea019a3e0 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9000de70 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9001d920 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.195+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea0102350 con 0x7f5ea006c580 2024-04-26T18:22:02.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.196+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea0104b80 con 0x7f5ea006c580 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.197+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5e9001dac0 con 0x7f5ea006c580 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.198+0000 7f5e977fe700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 0x7f5e8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.198+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f5e900d48f0 con 0x7f5ea006c580 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.198+0000 7f5ea64f9700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 0x7f5e8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.199+0000 7f5ea64f9700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 0x7f5e8c0a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c003800 tx=0x7f5e9c009f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.202+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e90058ff0 con 0x7f5ea006c580 2024-04-26T18:22:02.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:02 smithi170 ceph-mon[32697]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.470+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5ea002e490 con 0x7f5ea006c580 2024-04-26T18:22:02.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.475+0000 7f5e977fe700 1 -- 172.21.15.37:0/3717389822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7f5e900a1260 con 0x7f5ea006c580 2024-04-26T18:22:02.479 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 msgr2=0x7f5e8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 0x7f5e8c0a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c003800 tx=0x7f5e9c009f70 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 msgr2=0x7f5ea019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000bf20 tx=0x7f5e9000dc30 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 shutdown_connections 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5e8c0a3cd0 0x7f5e8c0a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ea0109490 0x7f5ea0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ea006bbc0 0x7f5ea019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 --2- 172.21.15.37:0/3717389822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea006c580 0x7f5ea019fd10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:02.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 >> 172.21.15.37:0/3717389822 conn(0x7f5ea0078390 msgr2=0x7f5ea01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 shutdown_connections 2024-04-26T18:22:02.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:02.478+0000 7f5ea875d700 1 -- 172.21.15.37:0/3717389822 wait complete. 2024-04-26T18:22:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:02 smithi120 ceph-mon[32244]: pgmap v322: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:03.125 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":813673186},{"type":"v1","addr":"172.21.15.37:6801","nonce":813673186}]},"active_addr":"172.21.15.37:6801/813673186","active_change":"2024-04-26T18:13:56.574585+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3994756262}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2586849717}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1405864097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3120809922}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3537975470}]}]} 2024-04-26T18:22:03.131 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T18:22:03.131 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T18:22:03.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:22:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:03 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3717389822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:22:03.384 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:03 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3717389822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:22:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:03 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3717389822' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:22:04.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:04 smithi170 ceph-mon[32697]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:04.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:04 smithi037 ceph-mon[29171]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:04 smithi120 ceph-mon[32244]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 -- 172.21.15.37:0/239463903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c108390 msgr2=0x7f9a7c108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 --2- 172.21.15.37:0/239463903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c108390 0x7f9a7c108770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009990 tx=0x7f9a64009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 -- 172.21.15.37:0/239463903 shutdown_connections 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 --2- 172.21.15.37:0/239463903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 --2- 172.21.15.37:0/239463903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c108390 0x7f9a7c108770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 --2- 172.21.15.37:0/239463903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c102320 0x7f9a7c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 -- 172.21.15.37:0/239463903 >> 172.21.15.37:0/239463903 conn(0x7f9a7c0792d0 msgr2=0x7f9a7c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 -- 172.21.15.37:0/239463903 shutdown_connections 2024-04-26T18:22:04.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.981+0000 7f9a830b5700 1 -- 172.21.15.37:0/239463903 wait complete. 2024-04-26T18:22:04.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 Processor -- start 2024-04-26T18:22:04.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 -- start start 2024-04-26T18:22:04.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c108390 0x7f9a7c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c1124e0 con 0x7f9a7c102320 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c112360 con 0x7f9a7c102ce0 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.982+0000 7f9a830b5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c112660 con 0x7f9a7c108390 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a80e51700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a80e51700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33750/0 (socket says 172.21.15.37:33750) 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a80e51700 1 -- 172.21.15.37:0/3774258345 learned_addr learned my addr 172.21.15.37:0/3774258345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a81652700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c108390 0x7f9a7c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 -- 172.21.15.37:0/3774258345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c108390 msgr2=0x7f9a7c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c108390 0x7f9a7c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 -- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 msgr2=0x7f9a7c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 -- 172.21.15.37:0/3774258345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c19a070 con 0x7f9a7c102ce0 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a80e51700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a7bfff700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c19fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e9f0 tx=0x7f9a6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c004220 con 0x7f9a7c102ce0 2024-04-26T18:22:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c19a360 con 0x7f9a7c102ce0 2024-04-26T18:22:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c0043c0 con 0x7f9a7c102ce0 2024-04-26T18:22:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.983+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c005020 con 0x7f9a7c102ce0 2024-04-26T18:22:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.984+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c1a6620 con 0x7f9a7c102ce0 2024-04-26T18:22:04.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.985+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c06a110 con 0x7f9a7c102ce0 2024-04-26T18:22:04.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.985+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a6c004750 con 0x7f9a7c102ce0 2024-04-26T18:22:04.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.986+0000 7f9a79ffb700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 0x7f9a680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.986+0000 7f9a80e51700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 0x7f9a680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.986+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f9a6c0d04f0 con 0x7f9a7c102ce0 2024-04-26T18:22:04.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.987+0000 7f9a80e51700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 0x7f9a680a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b2e0 tx=0x7f9a6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:04.991+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a6c09cd50 con 0x7f9a7c102ce0 2024-04-26T18:22:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.203+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a7c1a6850 con 0x7f9a7c102ce0 2024-04-26T18:22:05.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.204+0000 7f9a79ffb700 1 -- 172.21.15.37:0/3774258345 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7f9a6c09c650 con 0x7f9a7c102ce0 2024-04-26T18:22:05.205 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:22:05.205 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:21:58.040454+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.206+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 msgr2=0x7f9a680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.206+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 0x7f9a680a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b2e0 tx=0x7f9a6401a040 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.206+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 msgr2=0x7f9a7c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.206+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c19fc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e9f0 tx=0x7f9a6c00ed00 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 shutdown_connections 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3cd0 0x7f9a680a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c108390 0x7f9a7c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102320 0x7f9a7c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 --2- 172.21.15.37:0/3774258345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102ce0 0x7f9a7c19fc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 >> 172.21.15.37:0/3774258345 conn(0x7f9a7c0792d0 msgr2=0x7f9a7c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 shutdown_connections 2024-04-26T18:22:05.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:05.207+0000 7f9a830b5700 1 -- 172.21.15.37:0/3774258345 wait complete. 2024-04-26T18:22:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:05 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:05.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:05 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:05 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:05.832 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T18:22:05.833 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:22:06.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:06 smithi170 ceph-mon[32697]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 4 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:06.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:06 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3774258345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:06.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:06 smithi037 ceph-mon[29171]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 4 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:06.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:06 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3774258345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:06.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:06 smithi120 ceph-mon[32244]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 4 op/s; 159 B/s, 1 objects/s recovering 2024-04-26T18:22:06.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:06 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3774258345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:07 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:07 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:07.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:07 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 -- 172.21.15.37:0/1426464907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90006b9c0 msgr2=0x7fc90006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 --2- 172.21.15.37:0/1426464907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90006b9c0 0x7fc90006be20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009990 tx=0x7fc8ec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 -- 172.21.15.37:0/1426464907 shutdown_connections 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 --2- 172.21.15.37:0/1426464907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 0x7fc9001102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 --2- 172.21.15.37:0/1426464907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc900109490 0x7fc900109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 --2- 172.21.15.37:0/1426464907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90006b9c0 0x7fc90006be20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 -- 172.21.15.37:0/1426464907 >> 172.21.15.37:0/1426464907 conn(0x7fc900079290 msgr2=0x7fc9000796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:08.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 -- 172.21.15.37:0/1426464907 shutdown_connections 2024-04-26T18:22:08.462 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:08 smithi037 ceph-mon[29171]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 530 B/s wr, 6 op/s; 132 B/s, 1 objects/s recovering 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.460+0000 7fc906ecb700 1 -- 172.21.15.37:0/1426464907 wait complete. 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.461+0000 7fc906ecb700 1 Processor -- start 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.461+0000 7fc906ecb700 1 -- start start 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.461+0000 7fc906ecb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc90006b9c0 0x7fc90019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.461+0000 7fc906ecb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 0x7fc90019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc906ecb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:08.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc906ecb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9001135a0 con 0x7fc900109490 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc906ecb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc900113420 con 0x7fc90006c360 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc906ecb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc900113720 con 0x7fc90006b9c0 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33760/0 (socket says 172.21.15.37:33760) 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 -- 172.21.15.37:0/686256913 learned_addr learned my addr 172.21.15.37:0/686256913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc8fffff700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 0x7fc90019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc904c67700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc90006b9c0 0x7fc90019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 -- 172.21.15.37:0/686256913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc90006b9c0 msgr2=0x7fc90019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc90006b9c0 0x7fc90019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 -- 172.21.15.37:0/686256913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 msgr2=0x7fc90019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:08.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 0x7fc90019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 -- 172.21.15.37:0/686256913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc900199fe0 con 0x7fc900109490 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.462+0000 7fc905468700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007d70 tx=0x7fc8f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.463+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8016020 con 0x7fc900109490 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.463+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90019a2d0 con 0x7fc900109490 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.463+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f800ac90 con 0x7fc900109490 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.463+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80115e0 con 0x7fc900109490 2024-04-26T18:22:08.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.463+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a6590 con 0x7fc900109490 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.464+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90006a110 con 0x7fc900109490 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.465+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc8f8019020 con 0x7fc900109490 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.466+0000 7fc8fdffb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 0x7fc8f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.466+0000 7fc904c67700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 0x7fc8f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.466+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fc8f80d03c0 con 0x7fc900109490 2024-04-26T18:22:08.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.467+0000 7fc904c67700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 0x7fc8f00a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc90019a9c0 tx=0x7fc8f400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:08.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.471+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8f8020020 con 0x7fc900109490 2024-04-26T18:22:08.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.695+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc90002e490 con 0x7fc900109490 2024-04-26T18:22:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.698+0000 7fc8fdffb700 1 -- 172.21.15.37:0/686256913 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22650 (secure 0 0 0) 0x7fc8f809cd30 con 0x7fc900109490 2024-04-26T18:22:08.698 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:22:08.698 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:21:58.040454+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:22:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 msgr2=0x7fc8f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 0x7fc8f00a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc90019a9c0 tx=0x7fc8f400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 msgr2=0x7fc900199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007d70 tx=0x7fc8f800a340 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 shutdown_connections 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc8f00a3d50 0x7fc8f00a6210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc900109490 0x7fc900199880 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc90006b9c0 0x7fc90019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 --2- 172.21.15.37:0/686256913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90006c360 0x7fc90019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.700+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 >> 172.21.15.37:0/686256913 conn(0x7fc900079290 msgr2=0x7fc9001003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.701+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 shutdown_connections 2024-04-26T18:22:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:08.701+0000 7fc906ecb700 1 -- 172.21.15.37:0/686256913 wait complete. 2024-04-26T18:22:08.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:08 smithi120 ceph-mon[32244]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 530 B/s wr, 6 op/s; 132 B/s, 1 objects/s recovering 2024-04-26T18:22:08.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:08 smithi170 ceph-mon[32697]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 530 B/s wr, 6 op/s; 132 B/s, 1 objects/s recovering 2024-04-26T18:22:09.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T18:22:09.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T18:22:09.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T18:22:09.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T18:22:09.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T18:22:09.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T18:22:09.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T18:22:09.367 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T18:22:09.367 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T18:22:09.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T18:22:09.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T18:22:09.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T18:22:09.679 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:09.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:09 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:09.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:09 smithi037 ceph-mon[29171]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 480 B/s wr, 5 op/s; 119 B/s, 1 objects/s recovering 2024-04-26T18:22:09.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:09 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/686256913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:10.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:10.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:09 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:10.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:09 smithi120 ceph-mon[32244]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 480 B/s wr, 5 op/s; 119 B/s, 1 objects/s recovering 2024-04-26T18:22:10.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:09 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/686256913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:10.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:10.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:09 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:10.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:09 smithi170 ceph-mon[32697]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 480 B/s wr, 5 op/s; 119 B/s, 1 objects/s recovering 2024-04-26T18:22:10.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:09 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/686256913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:22:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.266 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.274 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.275 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.275 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.275 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.275 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.276 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:10.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:10.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:10.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:10 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:11.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:11.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:11.005 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:10 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:11.078 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:11.079 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:11.079 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:10 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:22:11.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:11 smithi037 ceph-mon[29171]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s; 96 B/s, 0 objects/s recovering 2024-04-26T18:22:12.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:11 smithi120 ceph-mon[32244]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s; 96 B/s, 0 objects/s recovering 2024-04-26T18:22:12.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:11 smithi170 ceph-mon[32697]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s; 96 B/s, 0 objects/s recovering 2024-04-26T18:22:13.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:13 smithi037 ceph-mon[29171]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:14.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:13 smithi120 ceph-mon[32244]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:14.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:13 smithi170 ceph-mon[32697]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:15.881 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:15 smithi037 ceph-mon[29171]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:16.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:15 smithi120 ceph-mon[32244]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:16.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:15 smithi170 ceph-mon[32697]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:17.055 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:17.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:17 smithi037 ceph-mon[29171]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:18.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:17 smithi120 ceph-mon[32244]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:18.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:17 smithi170 ceph-mon[32697]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 -- 172.21.15.37:0/1932885283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 msgr2=0x7fe36010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 --2- 172.21.15.37:0/1932885283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe36010f210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b0d0 tx=0x7fe35800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 -- 172.21.15.37:0/1932885283 shutdown_connections 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 --2- 172.21.15.37:0/1932885283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe36010f210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 --2- 172.21.15.37:0/1932885283 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3601025b0 0x7fe360102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 --2- 172.21.15.37:0/1932885283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe360102f60 0x7fe360106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 -- 172.21.15.37:0/1932885283 >> 172.21.15.37:0/1932885283 conn(0x7fe3600792d0 msgr2=0x7fe3600796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 -- 172.21.15.37:0/1932885283 shutdown_connections 2024-04-26T18:22:19.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.529+0000 7fe366a41700 1 -- 172.21.15.37:0/1932885283 wait complete. 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.530+0000 7fe366a41700 1 Processor -- start 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.530+0000 7fe366a41700 1 -- start start 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3601025b0 0x7fe36019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe360102f60 0x7fe36019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe360199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe360112620 con 0x7fe360107640 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3601124a0 con 0x7fe360102f60 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3601127a0 con 0x7fe3601025b0 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe360199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe360199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49698/0 (socket says 172.21.15.37:49698) 2024-04-26T18:22:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 -- 172.21.15.37:0/3713381978 learned_addr learned my addr 172.21.15.37:0/3713381978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe35ffff700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3601025b0 0x7fe36019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe35f7fe700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe360102f60 0x7fe36019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 -- 172.21.15.37:0/3713381978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3601025b0 msgr2=0x7fe36019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3601025b0 0x7fe36019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 -- 172.21.15.37:0/3713381978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe360102f60 msgr2=0x7fe36019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe360102f60 0x7fe36019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36019a1f0 con 0x7fe360107640 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe364fde700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe360199a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe3580079a0 tx=0x7fe3580079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35801f070 con 0x7fe360107640 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe358005070 con 0x7fe360107640 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36019a480 con 0x7fe360107640 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.531+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35801b680 con 0x7fe360107640 2024-04-26T18:22:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.532+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3601a67a0 con 0x7fe360107640 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.532+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe360063bb0 con 0x7fe360107640 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.534+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe358022070 con 0x7fe360107640 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.534+0000 7fe35d7fa700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe3480a3d50 0x7fe3480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.534+0000 7fe35ffff700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe3480a3d50 0x7fe3480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.534+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fe35800f020 con 0x7fe360107640 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.534+0000 7fe35d7fa700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] conn(0x7fe3480ad990 0x7fe3480afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.535+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3580020b0 con 0x7fe3480ad990 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.535+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe3580d1d80 con 0x7fe360107640 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.535+0000 7fe35f7fe700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] conn(0x7fe3480ad990 0x7fe3480afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.535+0000 7fe35ffff700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe3480a3d50 0x7fe3480a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe36019abd0 tx=0x7fe350009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.535+0000 7fe35f7fe700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] conn(0x7fe3480ad990 0x7fe3480afe30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.536+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== osd.3 v2:172.21.15.37:6810/3804676062 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3580020b0 con 0x7fe3480ad990 2024-04-26T18:22:19.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.553+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 --> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe360063fb0 con 0x7fe3480ad990 2024-04-26T18:22:19.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe35d7fa700 1 -- 172.21.15.37:0/3713381978 <== osd.3 v2:172.21.15.37:6810/3804676062 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe360063fb0 con 0x7fe3480ad990 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] conn(0x7fe3480ad990 msgr2=0x7fe3480afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] conn(0x7fe3480ad990 0x7fe3480afe30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe3480a3d50 msgr2=0x7fe3480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe3480a3d50 0x7fe3480a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe36019abd0 tx=0x7fe350009190 comp rx=0 tx=0).stop 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 msgr2=0x7fe360199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 --2- 172.21.15.37:0/3713381978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe360107640 0x7fe360199a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe3580079a0 tx=0x7fe3580079d0 comp rx=0 tx=0).stop 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe364fde700 1 -- 172.21.15.37:0/3713381978 reap_dead start 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 shutdown_connections 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 >> 172.21.15.37:0/3713381978 conn(0x7fe3600792d0 msgr2=0x7fe36006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 shutdown_connections 2024-04-26T18:22:19.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:19.554+0000 7fe366a41700 1 -- 172.21.15.37:0/3713381978 wait complete. 2024-04-26T18:22:19.967 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:19 smithi037 ceph-mon[29171]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:20.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:19 smithi120 ceph-mon[32244]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:20.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:19 smithi170 ceph-mon[32697]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:20.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/3467062976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18108390 msgr2=0x7f4f18108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/3467062976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18108390 0x7f4f18108770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009990 tx=0x7f4f08009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/3467062976 shutdown_connections 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/3467062976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f18102ce0 0x7f4f1810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/3467062976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18108390 0x7f4f18108770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/3467062976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f18102320 0x7f4f181027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/3467062976 >> 172.21.15.37:0/3467062976 conn(0x7f4f180792d0 msgr2=0x7f4f180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/3467062976 shutdown_connections 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/3467062976 wait complete. 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.527+0000 7f4f1e1a5700 1 Processor -- start 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 -- start start 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f18102320 0x7f4f1819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 0x7f4f1819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:20.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f18108390 0x7f4f181976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181113f0 con 0x7f4f18102ce0 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18111270 con 0x7f4f18108390 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f1e1a5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18111570 con 0x7f4f18102320 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 0x7f4f1819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 0x7f4f1819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49718/0 (socket says 172.21.15.37:49718) 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 -- 172.21.15.37:0/1271309221 learned_addr learned my addr 172.21.15.37:0/1271309221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f17fff700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f18108390 0x7f4f181976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f177fe700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f18102320 0x7f4f1819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 -- 172.21.15.37:0/1271309221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f18102320 msgr2=0x7f4f1819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f18102320 0x7f4f1819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 -- 172.21.15.37:0/1271309221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f18108390 msgr2=0x7f4f181976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f18108390 0x7f4f181976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f18197fb0 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f16ffd700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 0x7f4f1819db40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c010b80 tx=0x7f4f0c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.528+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01a030 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.529+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c00ede0 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.529+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c005280 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.529+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181982a0 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.529+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a4580 con 0x7f4f18102ce0 2024-04-26T18:22:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.529+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f18063b70 con 0x7f4f18102ce0 2024-04-26T18:22:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.531+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4f0c01d070 con 0x7f4f18102ce0 2024-04-26T18:22:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.531+0000 7f4f14ff9700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4f000a3cd0 0x7f4f000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.531+0000 7f4f177fe700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4f000a3cd0 0x7f4f000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f4f0c0d2ea0 con 0x7f4f18102ce0 2024-04-26T18:22:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f14ff9700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] conn(0x7f4f000ad910 0x7f4f000afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f000b0480 con 0x7f4f000ad910 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c0d32f0 con 0x7f4f18102ce0 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f17fff700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] conn(0x7f4f000ad910 0x7f4f000afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f177fe700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4f000a3cd0 0x7f4f000a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006010 tx=0x7f4f0801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.532+0000 7f4f17fff700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] conn(0x7f4f000ad910 0x7f4f000afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:20.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.533+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== osd.0 v2:172.21.15.37:6802/580247654 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f000b0480 con 0x7f4f000ad910 2024-04-26T18:22:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.549+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 --> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f18063ff0 con 0x7f4f000ad910 2024-04-26T18:22:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f14ff9700 1 -- 172.21.15.37:0/1271309221 <== osd.0 v2:172.21.15.37:6802/580247654 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f18063ff0 con 0x7f4f000ad910 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] conn(0x7f4f000ad910 msgr2=0x7f4f000afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] conn(0x7f4f000ad910 0x7f4f000afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4f000a3cd0 msgr2=0x7f4f000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4f000a3cd0 0x7f4f000a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006010 tx=0x7f4f0801a040 comp rx=0 tx=0).stop 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 msgr2=0x7f4f1819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 --2- 172.21.15.37:0/1271309221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f18102ce0 0x7f4f1819db40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c010b80 tx=0x7f4f0c010f40 comp rx=0 tx=0).stop 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f17fff700 1 -- 172.21.15.37:0/1271309221 reap_dead start 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 shutdown_connections 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 >> 172.21.15.37:0/1271309221 conn(0x7f4f180792d0 msgr2=0x7f4f18101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 shutdown_connections 2024-04-26T18:22:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:20.550+0000 7f4f1e1a5700 1 -- 172.21.15.37:0/1271309221 wait complete. 2024-04-26T18:22:21.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:21 smithi037 ceph-mon[29171]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:22:22.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:21 smithi120 ceph-mon[32244]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:22:22.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:21 smithi170 ceph-mon[32697]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:22:23.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:23 smithi037 ceph-mon[29171]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:24.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:23 smithi120 ceph-mon[32244]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:24.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:23 smithi170 ceph-mon[32697]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:24.473 INFO:teuthology.orchestra.run.smithi037.stdout:34359738453 2024-04-26T18:22:24.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T18:22:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 -- 172.21.15.37:0/1442953426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 msgr2=0x7f49d41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/1442953426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d41038f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009990 tx=0x7f49c4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 -- 172.21.15.37:0/1442953426 shutdown_connections 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/1442953426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49d4103e30 0x7f49d4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/1442953426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49d41094c0 0x7f49d41098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/1442953426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d41038f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 -- 172.21.15.37:0/1442953426 >> 172.21.15.37:0/1442953426 conn(0x7f49d407a580 msgr2=0x7f49d407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 -- 172.21.15.37:0/1442953426 shutdown_connections 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.310+0000 7f49dc1b4700 1 -- 172.21.15.37:0/1442953426 wait complete. 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 Processor -- start 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 -- start start 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49d4103e30 0x7f49d4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49d41094c0 0x7f49d40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41135c0 con 0x7f49d4103470 2024-04-26T18:22:25.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4113440 con 0x7f49d4103e30 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49dc1b4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4113740 con 0x7f49d41094c0 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d4075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41644/0 (socket says 172.21.15.37:41644) 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 -- 172.21.15.37:0/114887145 learned_addr learned my addr 172.21.15.37:0/114887145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d974f700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49d4103e30 0x7f49d4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49da751700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49d41094c0 0x7f49d40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 -- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49d41094c0 msgr2=0x7f49d40761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49d41094c0 0x7f49d40761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 -- 172.21.15.37:0/114887145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49d4103e30 msgr2=0x7f49d4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49d4103e30 0x7f49d4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41a8060 con 0x7f49d4103470 2024-04-26T18:22:25.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.311+0000 7f49d9f50700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d4075720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d950 tx=0x7f49d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.312+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0016070 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.312+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41a8350 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.312+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49d000fd40 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.312+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d000e3b0 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.312+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a8860 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.313+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49d4063b30 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.314+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f49d0019070 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.314+0000 7f49caffd700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49c00a3cd0 0x7f49c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.314+0000 7f49d974f700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49c00a3cd0 0x7f49c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f49d00d0490 con 0x7f49d4103470 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49caffd700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] conn(0x7f49c00ad910 0x7f49c00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49d0013ec0 con 0x7f49c00ad910 2024-04-26T18:22:25.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f49d00d08e0 con 0x7f49d4103470 2024-04-26T18:22:25.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49d974f700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49c00a3cd0 0x7f49c00a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b2e0 tx=0x7f49c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.315+0000 7f49da751700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] conn(0x7f49c00ad910 0x7f49c00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.316+0000 7f49da751700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] conn(0x7f49c00ad910 0x7f49c00afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.316+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== osd.8 v2:172.21.15.170:6816/428251464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49d0013ec0 con 0x7f49c00ad910 2024-04-26T18:22:25.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.333+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 --> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49d4063f30 con 0x7f49c00ad910 2024-04-26T18:22:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49caffd700 1 -- 172.21.15.37:0/114887145 <== osd.8 v2:172.21.15.170:6816/428251464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49d4063f30 con 0x7f49c00ad910 2024-04-26T18:22:25.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] conn(0x7f49c00ad910 msgr2=0x7f49c00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] conn(0x7f49c00ad910 0x7f49c00afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49c00a3cd0 msgr2=0x7f49c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f49c00a3cd0 0x7f49c00a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b2e0 tx=0x7f49c4005960 comp rx=0 tx=0).stop 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 msgr2=0x7f49d4075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 --2- 172.21.15.37:0/114887145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4103470 0x7f49d4075720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f49d000d950 tx=0x7f49d000dc60 comp rx=0 tx=0).stop 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49da751700 1 -- 172.21.15.37:0/114887145 reap_dead start 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 shutdown_connections 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 >> 172.21.15.37:0/114887145 conn(0x7f49d407a580 msgr2=0x7f49d4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 shutdown_connections 2024-04-26T18:22:25.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.334+0000 7f49dc1b4700 1 -- 172.21.15.37:0/114887145 wait complete. 2024-04-26T18:22:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 -- 172.21.15.37:0/4144892180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806bad0 msgr2=0x7f6bb81102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 --2- 172.21.15.37:0/4144892180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806bad0 0x7f6bb81102e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000b0a0 tx=0x7f6bb000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 -- 172.21.15.37:0/4144892180 shutdown_connections 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 --2- 172.21.15.37:0/4144892180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806bad0 0x7f6bb81102e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 --2- 172.21.15.37:0/4144892180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb81094a0 0x7f6bb8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 --2- 172.21.15.37:0/4144892180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bb806b130 0x7f6bb806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 -- 172.21.15.37:0/4144892180 >> 172.21.15.37:0/4144892180 conn(0x7f6bb80792d0 msgr2=0x7f6bb80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 -- 172.21.15.37:0/4144892180 shutdown_connections 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.216+0000 7f6bbe328700 1 -- 172.21.15.37:0/4144892180 wait complete. 2024-04-26T18:22:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 Processor -- start 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 -- start start 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 0x7f6bb819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb806bad0 0x7f6bb819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bb81094a0 0x7f6bb81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81135f0 con 0x7f6bb806b130 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8113470 con 0x7f6bb81094a0 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bbe328700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8113770 con 0x7f6bb806bad0 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bb7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 0x7f6bb819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bb7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 0x7f6bb819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41628/0 (socket says 172.21.15.37:41628) 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.217+0000 7f6bb7fff700 1 -- 172.21.15.37:0/2239586009 learned_addr learned my addr 172.21.15.37:0/2239586009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bbc8c5700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bb81094a0 0x7f6bb81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb77fe700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb806bad0 0x7f6bb819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 -- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb806bad0 msgr2=0x7f6bb819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6bb806bad0 0x7f6bb819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 -- 172.21.15.37:0/2239586009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bb81094a0 msgr2=0x7f6bb81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6bb81094a0 0x7f6bb81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb819a010 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb7fff700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 0x7f6bb819f6a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c9b0 tx=0x7f6ba800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8007b00 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb819a300 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba8007ca0 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8018730 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.218+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a6570 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.219+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bb8063af0 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.220+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ba800f450 con 0x7f6bb806b130 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb57fa700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ba00a3cd0 0x7f6ba00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb77fe700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ba00a3cd0 0x7f6ba00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f6ba80d0c80 con 0x7f6bb806b130 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb57fa700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] conn(0x7f6ba00ad910 0x7f6ba00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ba8014ec0 con 0x7f6ba00ad910 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6ba80d1030 con 0x7f6bb806b130 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bb77fe700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ba00a3cd0 0x7f6ba00a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bac005ea0 tx=0x7f6bac005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.221+0000 7f6bbc8c5700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] conn(0x7f6ba00ad910 0x7f6ba00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.222+0000 7f6bbc8c5700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] conn(0x7f6ba00ad910 0x7f6ba00afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.223+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== osd.2 v2:172.21.15.170:6800/1287805124 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ba8014ec0 con 0x7f6ba00ad910 2024-04-26T18:22:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.239+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 --> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bb8063f20 con 0x7f6ba00ad910 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bb57fa700 1 -- 172.21.15.37:0/2239586009 <== osd.2 v2:172.21.15.170:6800/1287805124 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6bb8063f20 con 0x7f6ba00ad910 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] conn(0x7f6ba00ad910 msgr2=0x7f6ba00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] conn(0x7f6ba00ad910 0x7f6ba00afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ba00a3cd0 msgr2=0x7f6ba00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6ba00a3cd0 0x7f6ba00a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bac005ea0 tx=0x7f6bac005e30 comp rx=0 tx=0).stop 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 msgr2=0x7f6bb819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbe328700 1 --2- 172.21.15.37:0/2239586009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bb806b130 0x7f6bb819f6a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800c9b0 tx=0x7f6ba800cd70 comp rx=0 tx=0).stop 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.240+0000 7f6bbc8c5700 1 -- 172.21.15.37:0/2239586009 reap_dead start 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.241+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 shutdown_connections 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.241+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 >> 172.21.15.37:0/2239586009 conn(0x7f6bb80792d0 msgr2=0x7f6bb8100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.241+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 shutdown_connections 2024-04-26T18:22:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:25.241+0000 7f6bbe328700 1 -- 172.21.15.37:0/2239586009 wait complete. 2024-04-26T18:22:25.967 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:25 smithi037 ceph-mon[29171]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:26.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:25 smithi120 ceph-mon[32244]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:26.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:25 smithi170 ceph-mon[32697]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 -- 172.21.15.37:0/74515053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c06b130 msgr2=0x7f1a7c06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 --2- 172.21.15.37:0/74515053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c06b130 0x7f1a7c06b590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 -- 172.21.15.37:0/74515053 shutdown_connections 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 --2- 172.21.15.37:0/74515053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a7c06bad0 0x7f1a7c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 --2- 172.21.15.37:0/74515053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 --2- 172.21.15.37:0/74515053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c06b130 0x7f1a7c06b590 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 -- 172.21.15.37:0/74515053 >> 172.21.15.37:0/74515053 conn(0x7f1a7c0792d0 msgr2=0x7f1a7c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 -- 172.21.15.37:0/74515053 shutdown_connections 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 -- 172.21.15.37:0/74515053 wait complete. 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.590+0000 7f1a815a2700 1 Processor -- start 2024-04-26T18:22:26.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 -- start start 2024-04-26T18:22:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a7c06b130 0x7f1a7c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c06bad0 0x7f1a7c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c113610 con 0x7f1a7c1094a0 2024-04-26T18:22:26.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c113490 con 0x7f1a7c06b130 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a815a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c113790 con 0x7f1a7c06bad0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41658/0 (socket says 172.21.15.37:41658) 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 -- 172.21.15.37:0/159755083 learned_addr learned my addr 172.21.15.37:0/159755083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7a7fc700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c06bad0 0x7f1a7c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7affd700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a7c06b130 0x7f1a7c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 -- 172.21.15.37:0/159755083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c06bad0 msgr2=0x7f1a7c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c06bad0 0x7f1a7c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 -- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a7c06b130 msgr2=0x7f1a7c1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a7c06b130 0x7f1a7c1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7b7fe700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c1a8070 con 0x7f1a7c1094a0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.591+0000 7f1a7a7fc700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a7c06bad0 0x7f1a7c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a7b7fe700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c102e70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007d70 tx=0x7f1a7400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:26.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74016020 con 0x7f1a7c1094a0 2024-04-26T18:22:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c1a8360 con 0x7f1a7c1094a0 2024-04-26T18:22:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7400ad40 con 0x7f1a7c1094a0 2024-04-26T18:22:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a740115e0 con 0x7f1a7c1094a0 2024-04-26T18:22:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.592+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a8850 con 0x7f1a7c1094a0 2024-04-26T18:22:26.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.593+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a7c063b30 con 0x7f1a7c1094a0 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.594+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1a74019020 con 0x7f1a7c1094a0 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.594+0000 7f1a63fff700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1a640a3cd0 0x7f1a640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a7affd700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1a640a3cd0 0x7f1a640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1a740d0a60 con 0x7f1a7c1094a0 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a63fff700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] conn(0x7f1a640ad910 0x7f1a640afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a640b0480 con 0x7f1a640ad910 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1a740d0eb0 con 0x7f1a7c1094a0 2024-04-26T18:22:26.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a7affd700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1a640a3cd0 0x7f1a640a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006fd0 tx=0x7f1a6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.595+0000 7f1a7a7fc700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] conn(0x7f1a640ad910 0x7f1a640afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.596+0000 7f1a7a7fc700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] conn(0x7f1a640ad910 0x7f1a640afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.597+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== osd.10 v2:172.21.15.120:6824/2391866955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a640b0480 con 0x7f1a640ad910 2024-04-26T18:22:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.616+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 --> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a7c063fb0 con 0x7f1a640ad910 2024-04-26T18:22:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a63fff700 1 -- 172.21.15.37:0/159755083 <== osd.10 v2:172.21.15.120:6824/2391866955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a7c063fb0 con 0x7f1a640ad910 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] conn(0x7f1a640ad910 msgr2=0x7f1a640afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] conn(0x7f1a640ad910 0x7f1a640afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1a640a3cd0 msgr2=0x7f1a640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1a640a3cd0 0x7f1a640a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c006fd0 tx=0x7f1a6c008040 comp rx=0 tx=0).stop 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 msgr2=0x7f1a7c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a815a2700 1 --2- 172.21.15.37:0/159755083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a7c1094a0 0x7f1a7c102e70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007d70 tx=0x7f1a7400a340 comp rx=0 tx=0).stop 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.617+0000 7f1a7b7fe700 1 -- 172.21.15.37:0/159755083 reap_dead start 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.618+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 shutdown_connections 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.618+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 >> 172.21.15.37:0/159755083 conn(0x7f1a7c0792d0 msgr2=0x7f1a7c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.618+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 shutdown_connections 2024-04-26T18:22:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:26.618+0000 7f1a815a2700 1 -- 172.21.15.37:0/159755083 wait complete. 2024-04-26T18:22:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1875129991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb598102670 msgr2=0x7fb598102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1875129991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb598102670 0x7fb598102af0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0099c0 tx=0x7fb58c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1875129991 shutdown_connections 2024-04-26T18:22:27.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1875129991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598103030 0x7fb59810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1875129991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5981073a0 0x7fb598107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1875129991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb598102670 0x7fb598102af0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1875129991 >> 172.21.15.37:0/1875129991 conn(0x7fb598078190 msgr2=0x7fb5980785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1875129991 shutdown_connections 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.211+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1875129991 wait complete. 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 Processor -- start 2024-04-26T18:22:27.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 -- start start 2024-04-26T18:22:27.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb598102670 0x7fb598104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598103030 0x7fb598104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 0x7fb598104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb598112620 con 0x7fb5981073a0 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981124a0 con 0x7fb598103030 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.212+0000 7fb59e8d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981127a0 con 0x7fb598102670 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 0x7fb598104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 0x7fb598104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41674/0 (socket says 172.21.15.37:41674) 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 -- 172.21.15.37:0/1363726021 learned_addr learned my addr 172.21.15.37:0/1363726021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb597fff700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb598102670 0x7fb598104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb5977fe700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598103030 0x7fb598104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 -- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb598102670 msgr2=0x7fb598104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb598102670 0x7fb598104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 -- 172.21.15.37:0/1363726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598103030 msgr2=0x7fb598104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb598103030 0x7fb598104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981a8130 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59ce73700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 0x7fb598104ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb590007d70 tx=0x7fb59000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590016020 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981a8420 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb59000ac90 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5900115e0 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.213+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981a8930 con 0x7fb5981073a0 2024-04-26T18:22:27.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.214+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb598063b30 con 0x7fb5981073a0 2024-04-26T18:22:27.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.215+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb590019020 con 0x7fb5981073a0 2024-04-26T18:22:27.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb5957fa700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb5800a3cd0 0x7fb5800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:27.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb597fff700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb5800a3cd0 0x7fb5800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:27.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fb5900d0900 con 0x7fb5981073a0 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb5957fa700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] conn(0x7fb5800ad910 0x7fb5800afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb590024e50 con 0x7fb5800ad910 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb5900d0d50 con 0x7fb5981073a0 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.216+0000 7fb597fff700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb5800a3cd0 0x7fb5800a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb588009a10 tx=0x7fb588009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.217+0000 7fb5977fe700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] conn(0x7fb5800ad910 0x7fb5800afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.217+0000 7fb5977fe700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] conn(0x7fb5800ad910 0x7fb5800afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:27.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.218+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== osd.5 v2:172.21.15.170:6808/3652828620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb590024e50 con 0x7fb5800ad910 2024-04-26T18:22:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.237+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 --> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb598063f30 con 0x7fb5800ad910 2024-04-26T18:22:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.237+0000 7fb5957fa700 1 -- 172.21.15.37:0/1363726021 <== osd.5 v2:172.21.15.170:6808/3652828620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb598063f30 con 0x7fb5800ad910 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] conn(0x7fb5800ad910 msgr2=0x7fb5800afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] conn(0x7fb5800ad910 0x7fb5800afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb5800a3cd0 msgr2=0x7fb5800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb5800a3cd0 0x7fb5800a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb588009a10 tx=0x7fb588009190 comp rx=0 tx=0).stop 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 msgr2=0x7fb598104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 --2- 172.21.15.37:0/1363726021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5981073a0 0x7fb598104ad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb590007d70 tx=0x7fb59000a340 comp rx=0 tx=0).stop 2024-04-26T18:22:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59ce73700 1 -- 172.21.15.37:0/1363726021 reap_dead start 2024-04-26T18:22:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 shutdown_connections 2024-04-26T18:22:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 >> 172.21.15.37:0/1363726021 conn(0x7fb598078190 msgr2=0x7fb59806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 shutdown_connections 2024-04-26T18:22:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:27.238+0000 7fb59e8d6700 1 -- 172.21.15.37:0/1363726021 wait complete. 2024-04-26T18:22:27.664 INFO:teuthology.orchestra.run.smithi037.stdout:77309411404 2024-04-26T18:22:27.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T18:22:27.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:27 smithi037 ceph-mon[29171]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T18:22:28.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:27 smithi120 ceph-mon[32244]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T18:22:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:27 smithi170 ceph-mon[32697]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-26T18:22:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 -- 172.21.15.37:0/1531327057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 msgr2=0x7f613c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 --2- 172.21.15.37:0/1531327057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c1098c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f612c009990 tx=0x7f612c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 -- 172.21.15.37:0/1531327057 shutdown_connections 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 --2- 172.21.15.37:0/1531327057 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c103e40 0x7f613c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 --2- 172.21.15.37:0/1531327057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c1098c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 --2- 172.21.15.37:0/1531327057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c1034a0 0x7f613c103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 -- 172.21.15.37:0/1531327057 >> 172.21.15.37:0/1531327057 conn(0x7f613c07a5d0 msgr2=0x7f613c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 -- 172.21.15.37:0/1531327057 shutdown_connections 2024-04-26T18:22:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.649+0000 7f6142b76700 1 -- 172.21.15.37:0/1531327057 wait complete. 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 Processor -- start 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 -- start start 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c1034a0 0x7f613c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c103e40 0x7f613c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1135d0 con 0x7f613c1094e0 2024-04-26T18:22:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113450 con 0x7f613c1034a0 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6142b76700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113750 con 0x7f613c103e40 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6141113700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6141113700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c0761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41692/0 (socket says 172.21.15.37:41692) 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6141113700 1 -- 172.21.15.37:0/287968273 learned_addr learned my addr 172.21.15.37:0/287968273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f613bfff700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c103e40 0x7f613c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.650+0000 7f6140912700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c1034a0 0x7f613c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 -- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c103e40 msgr2=0x7f613c075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f613c103e40 0x7f613c075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 -- 172.21.15.37:0/287968273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c1034a0 msgr2=0x7f613c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c1034a0 0x7f613c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c1a8070 con 0x7f613c1094e0 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6140912700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f613c1034a0 0x7f613c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6141113700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c0761a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613400ab50 tx=0x7f613400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134017020 con 0x7f613c1094e0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6134004bb0 con 0x7f613c1094e0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c1a8360 con 0x7f613c1094e0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134005280 con 0x7f613c1094e0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.651+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1a88a0 con 0x7f613c1094e0 2024-04-26T18:22:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.652+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f613c063b30 con 0x7f613c1094e0 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.653+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f613400c620 con 0x7f613c1094e0 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6139ffb700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f61240a3cd0 0x7f61240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6140912700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f61240a3cd0 0x7f61240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f61340d0520 con 0x7f613c1094e0 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6139ffb700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] conn(0x7f61240ad910 0x7f61240afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6134023e50 con 0x7f61240ad910 2024-04-26T18:22:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.654+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f61340d08d0 con 0x7f613c1094e0 2024-04-26T18:22:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.655+0000 7f6140912700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f61240a3cd0 0x7f61240a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f612c006010 tx=0x7f612c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.655+0000 7f613bfff700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] conn(0x7f61240ad910 0x7f61240afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.655+0000 7f613bfff700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] conn(0x7f61240ad910 0x7f61240afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.656+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== osd.11 v2:172.21.15.170:6824/49311375 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6134023e50 con 0x7f61240ad910 2024-04-26T18:22:28.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.673+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 --> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f613c063fb0 con 0x7f61240ad910 2024-04-26T18:22:28.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.674+0000 7f6139ffb700 1 -- 172.21.15.37:0/287968273 <== osd.11 v2:172.21.15.170:6824/49311375 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f613c063fb0 con 0x7f61240ad910 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.674+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] conn(0x7f61240ad910 msgr2=0x7f61240afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.674+0000 7f6142b76700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] conn(0x7f61240ad910 0x7f61240afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.674+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f61240a3cd0 msgr2=0x7f61240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.674+0000 7f6142b76700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f61240a3cd0 0x7f61240a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f612c006010 tx=0x7f612c01a040 comp rx=0 tx=0).stop 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 msgr2=0x7f613c0761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 --2- 172.21.15.37:0/287968273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f613c1094e0 0x7f613c0761a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f613400ab50 tx=0x7f613400ae60 comp rx=0 tx=0).stop 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6141113700 1 -- 172.21.15.37:0/287968273 reap_dead start 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 shutdown_connections 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 >> 172.21.15.37:0/287968273 conn(0x7f613c07a5d0 msgr2=0x7f613c101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 shutdown_connections 2024-04-26T18:22:28.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:28.675+0000 7f6142b76700 1 -- 172.21.15.37:0/287968273 wait complete. 2024-04-26T18:22:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 -- 172.21.15.37:0/1724308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06c360 msgr2=0x7f15ac1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 --2- 172.21.15.37:0/1724308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06c360 0x7f15ac1102d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b0a0 tx=0x7f15a400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 -- 172.21.15.37:0/1724308519 shutdown_connections 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 --2- 172.21.15.37:0/1724308519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06c360 0x7f15ac1102d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 --2- 172.21.15.37:0/1724308519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15ac109490 0x7f15ac109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 --2- 172.21.15.37:0/1724308519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac06b9c0 0x7f15ac06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 -- 172.21.15.37:0/1724308519 >> 172.21.15.37:0/1724308519 conn(0x7f15ac079290 msgr2=0x7f15ac0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 -- 172.21.15.37:0/1724308519 shutdown_connections 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 -- 172.21.15.37:0/1724308519 wait complete. 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.365+0000 7f15b412b700 1 Processor -- start 2024-04-26T18:22:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 -- start start 2024-04-26T18:22:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 0x7f15ac1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15ac06c360 0x7f15ac102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac109490 0x7f15ac102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac113640 con 0x7f15ac06b9c0 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac1134c0 con 0x7f15ac06c360 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b412b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac1137c0 con 0x7f15ac109490 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 0x7f15ac1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 0x7f15ac1023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41722/0 (socket says 172.21.15.37:41722) 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 -- 172.21.15.37:0/1039727970 learned_addr learned my addr 172.21.15.37:0/1039727970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b16c6700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15ac06c360 0x7f15ac102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b26c8700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac109490 0x7f15ac102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 -- 172.21.15.37:0/1039727970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac109490 msgr2=0x7f15ac102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac109490 0x7f15ac102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 -- 172.21.15.37:0/1039727970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15ac06c360 msgr2=0x7f15ac102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f15ac06c360 0x7f15ac102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac1a8020 con 0x7f15ac06b9c0 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b26c8700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15ac109490 0x7f15ac102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.366+0000 7f15b1ec7700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 0x7f15ac1023e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d950 tx=0x7f15a800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.367+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8016070 con 0x7f15ac06b9c0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.367+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a8310 con 0x7f15ac06b9c0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.367+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8010460 con 0x7f15ac06b9c0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.367+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a800f620 con 0x7f15ac06b9c0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.367+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a88a0 con 0x7f15ac06b9c0 2024-04-26T18:22:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.368+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1590001030 con 0x7f15ac06b9c0 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.369+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f15a8019070 con 0x7f15ac06b9c0 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.369+0000 7f15a2ffd700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f15980a3cd0 0x7f15980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.369+0000 7f15b16c6700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f15980a3cd0 0x7f15980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.369+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f15a80d05f0 con 0x7f15ac06b9c0 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15a2ffd700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] conn(0x7f15980ad910 0x7f15980afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15a8013ec0 con 0x7f15980ad910 2024-04-26T18:22:29.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f15a80d0a40 con 0x7f15ac06b9c0 2024-04-26T18:22:29.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15b26c8700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] conn(0x7f15980ad910 0x7f15980afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15b16c6700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f15980a3cd0 0x7f15980a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.370+0000 7f15b26c8700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] conn(0x7f15980ad910 0x7f15980afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.371+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== osd.9 v2:172.21.15.37:6826/4227134134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15a8013ec0 con 0x7f15980ad910 2024-04-26T18:22:29.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 --> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1590002e60 con 0x7f15980ad910 2024-04-26T18:22:29.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15a2ffd700 1 -- 172.21.15.37:0/1039727970 <== osd.9 v2:172.21.15.37:6826/4227134134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1590002e60 con 0x7f15980ad910 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] conn(0x7f15980ad910 msgr2=0x7f15980afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] conn(0x7f15980ad910 0x7f15980afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f15980a3cd0 msgr2=0x7f15980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f15980a3cd0 0x7f15980a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).stop 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 msgr2=0x7f15ac1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b412b700 1 --2- 172.21.15.37:0/1039727970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15ac06b9c0 0x7f15ac1023e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d950 tx=0x7f15a800dd10 comp rx=0 tx=0).stop 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.388+0000 7f15b26c8700 1 -- 172.21.15.37:0/1039727970 reap_dead start 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.389+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 shutdown_connections 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.389+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 >> 172.21.15.37:0/1039727970 conn(0x7f15ac079290 msgr2=0x7f15ac1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.389+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 shutdown_connections 2024-04-26T18:22:29.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.389+0000 7f15b412b700 1 -- 172.21.15.37:0/1039727970 wait complete. 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 -- 172.21.15.37:0/2340416983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc260109490 msgr2=0x7fc260109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 --2- 172.21.15.37:0/2340416983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc260109490 0x7fc260109870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc250009990 tx=0x7fc250009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 -- 172.21.15.37:0/2340416983 shutdown_connections 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 --2- 172.21.15.37:0/2340416983 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc26006c360 0x7fc2601102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 --2- 172.21.15.37:0/2340416983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc260109490 0x7fc260109870 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 --2- 172.21.15.37:0/2340416983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc26006b9c0 0x7fc26006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 -- 172.21.15.37:0/2340416983 >> 172.21.15.37:0/2340416983 conn(0x7fc260079290 msgr2=0x7fc2600796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 -- 172.21.15.37:0/2340416983 shutdown_connections 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.236+0000 7fc265d53700 1 -- 172.21.15.37:0/2340416983 wait complete. 2024-04-26T18:22:29.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 Processor -- start 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 -- start start 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 0x7fc26019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc26006c360 0x7fc26019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc260109490 0x7fc260199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113580 con 0x7fc26006b9c0 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113400 con 0x7fc260109490 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc265d53700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc260113700 con 0x7fc26006c360 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 0x7fc26019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc25f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 0x7fc26019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41708/0 (socket says 172.21.15.37:41708) 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.237+0000 7fc25f7fe700 1 -- 172.21.15.37:0/2097684946 learned_addr learned my addr 172.21.15.37:0/2097684946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25effd700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc26006c360 0x7fc26019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25ffff700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc260109490 0x7fc260199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 -- 172.21.15.37:0/2097684946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc26006c360 msgr2=0x7fc26019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc26006c360 0x7fc26019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 -- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc260109490 msgr2=0x7fc260199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc260109490 0x7fc260199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc260199fe0 con 0x7fc26006b9c0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25ffff700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc260109490 0x7fc260199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25f7fe700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 0x7fc26019f680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc250000c00 tx=0x7fc2500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250027070 con 0x7fc26006b9c0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2500041d0 con 0x7fc26006b9c0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26019a270 con 0x7fc26006b9c0 2024-04-26T18:22:29.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000f460 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.238+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2601a64e0 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.239+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc260063bb0 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.240+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc25002a070 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25cff9700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2480a3cd0 0x7fc2480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25effd700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2480a3cd0 0x7fc2480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fc2500d1ce0 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25cff9700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] conn(0x7fc2480ad910 0x7fc2480afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2500043d0 con 0x7fc2480ad910 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc2500d2090 con 0x7fc26006b9c0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25effd700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2480a3cd0 0x7fc2480a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc254005ea0 tx=0x7fc254005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.241+0000 7fc25ffff700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] conn(0x7fc2480ad910 0x7fc2480afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.242+0000 7fc25ffff700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] conn(0x7fc2480ad910 0x7fc2480afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:29.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.243+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== osd.7 v2:172.21.15.120:6816/152962141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2500043d0 con 0x7fc2480ad910 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.260+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 --> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc260064030 con 0x7fc2480ad910 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.260+0000 7fc25cff9700 1 -- 172.21.15.37:0/2097684946 <== osd.7 v2:172.21.15.120:6816/152962141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc260064030 con 0x7fc2480ad910 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] conn(0x7fc2480ad910 msgr2=0x7fc2480afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] conn(0x7fc2480ad910 0x7fc2480afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2480a3cd0 msgr2=0x7fc2480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fc2480a3cd0 0x7fc2480a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc254005ea0 tx=0x7fc254005e30 comp rx=0 tx=0).stop 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 msgr2=0x7fc26019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 --2- 172.21.15.37:0/2097684946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc26006b9c0 0x7fc26019f680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc250000c00 tx=0x7fc2500047b0 comp rx=0 tx=0).stop 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc25ffff700 1 -- 172.21.15.37:0/2097684946 reap_dead start 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 shutdown_connections 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 >> 172.21.15.37:0/2097684946 conn(0x7fc260079290 msgr2=0x7fc2601003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 shutdown_connections 2024-04-26T18:22:29.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:29.261+0000 7fc265d53700 1 -- 172.21.15.37:0/2097684946 wait complete. 2024-04-26T18:22:29.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:29 smithi037 ceph-mon[29171]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T18:22:30.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:29 smithi120 ceph-mon[32244]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T18:22:30.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:29 smithi170 ceph-mon[32697]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T18:22:30.721 INFO:teuthology.orchestra.run.smithi037.stdout:287762808874 2024-04-26T18:22:30.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T18:22:31.138 INFO:teuthology.orchestra.run.smithi037.stdout:343597383712 2024-04-26T18:22:31.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T18:22:31.784 INFO:teuthology.orchestra.run.smithi037.stdout:219043332148 2024-04-26T18:22:31.784 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T18:22:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.973+0000 7f8f51989700 1 -- 172.21.15.37:0/3469063661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 msgr2=0x7f8f4c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.973+0000 7f8f51989700 1 --2- 172.21.15.37:0/3469063661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c110330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400b0d0 tx=0x7f8f4400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 -- 172.21.15.37:0/3469063661 shutdown_connections 2024-04-26T18:22:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 --2- 172.21.15.37:0/3469063661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c110330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 --2- 172.21.15.37:0/3469063661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f4c109560 0x7f8f4c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 --2- 172.21.15.37:0/3469063661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c1034c0 0x7f8f4c103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 -- 172.21.15.37:0/3469063661 >> 172.21.15.37:0/3469063661 conn(0x7f8f4c07a5d0 msgr2=0x7f8f4c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 -- 172.21.15.37:0/3469063661 shutdown_connections 2024-04-26T18:22:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 -- 172.21.15.37:0/3469063661 wait complete. 2024-04-26T18:22:31.975 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:31 smithi037 ceph-mon[29171]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.974+0000 7f8f51989700 1 Processor -- start 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 -- start start 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f4c1034c0 0x7f8f4c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 0x7f8f4c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c113640 con 0x7f8f4c103e80 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1134c0 con 0x7f8f4c1034c0 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f51989700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1137c0 con 0x7f8f4c109560 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4a7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 0x7f8f4c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4a7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41736/0 (socket says 172.21.15.37:41736) 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 0x7f8f4c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:49534/0 (socket says 172.21.15.37:49534) 2024-04-26T18:22:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4a7fc700 1 -- 172.21.15.37:0/1613521705 learned_addr learned my addr 172.21.15.37:0/1613521705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.975+0000 7f8f4affd700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f4c1034c0 0x7f8f4c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 -- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f4c1034c0 msgr2=0x7f8f4c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f4c1034c0 0x7f8f4c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 -- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 msgr2=0x7f8f4c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c19a020 con 0x7f8f4c109560 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4a7fc700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c103e80 0x7f8f4c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f4b7fe700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 0x7f8f4c199840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009fd0 tx=0x7f8f44007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44021020 con 0x7f8f4c109560 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f44007e40 con 0x7f8f4c109560 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c19a2b0 con 0x7f8f4c109560 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44005050 con 0x7f8f4c109560 2024-04-26T18:22:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.976+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c075650 con 0x7f8f4c109560 2024-04-26T18:22:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.977+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f4c063b30 con 0x7f8f4c109560 2024-04-26T18:22:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.979+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f44024020 con 0x7f8f4c109560 2024-04-26T18:22:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.979+0000 7f8f50987700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8f340a3d50 0x7f8f340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f8f4400f020 con 0x7f8f4c109560 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f4affd700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8f340a3d50 0x7f8f340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f50987700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] conn(0x7f8f340ad990 0x7f8f340afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f340b0500 con 0x7f8f340ad990 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8f440d2210 con 0x7f8f4c109560 2024-04-26T18:22:31.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f4a7fc700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] conn(0x7f8f340ad990 0x7f8f340afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:31.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.980+0000 7f8f4affd700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8f340a3d50 0x7f8f340a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005950 tx=0x7f8f3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:31.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.981+0000 7f8f4a7fc700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] conn(0x7f8f340ad990 0x7f8f340afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:31.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:31.982+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== osd.1 v2:172.21.15.120:6800/634623221 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f340b0500 con 0x7f8f340ad990 2024-04-26T18:22:31.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.000+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 --> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f4c063fb0 con 0x7f8f340ad990 2024-04-26T18:22:32.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.000+0000 7f8f50987700 1 -- 172.21.15.37:0/1613521705 <== osd.1 v2:172.21.15.120:6800/634623221 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8f4c063fb0 con 0x7f8f340ad990 2024-04-26T18:22:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] conn(0x7f8f340ad990 msgr2=0x7f8f340afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] conn(0x7f8f340ad990 0x7f8f340afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8f340a3d50 msgr2=0x7f8f340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8f340a3d50 0x7f8f340a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c005950 tx=0x7f8f3c00b410 comp rx=0 tx=0).stop 2024-04-26T18:22:32.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 msgr2=0x7f8f4c199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 --2- 172.21.15.37:0/1613521705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f4c109560 0x7f8f4c199840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009fd0 tx=0x7f8f44007a00 comp rx=0 tx=0).stop 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f4b7fe700 1 -- 172.21.15.37:0/1613521705 reap_dead start 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 shutdown_connections 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 >> 172.21.15.37:0/1613521705 conn(0x7f8f4c07a5d0 msgr2=0x7f8f4c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 shutdown_connections 2024-04-26T18:22:32.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.001+0000 7f8f51989700 1 -- 172.21.15.37:0/1613521705 wait complete. 2024-04-26T18:22:32.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:31 smithi120 ceph-mon[32244]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:32.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:31 smithi170 ceph-mon[32697]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:32.594 INFO:teuthology.orchestra.run.smithi037.stdout:253403070510 2024-04-26T18:22:32.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 -- 172.21.15.37:0/1869473646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 msgr2=0x7f7ca810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 --2- 172.21.15.37:0/1869473646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca810f1d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000b0a0 tx=0x7f7ca000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 -- 172.21.15.37:0/1869473646 shutdown_connections 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 --2- 172.21.15.37:0/1869473646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca810f1d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 --2- 172.21.15.37:0/1869473646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca8108390 0x7f7ca8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 --2- 172.21.15.37:0/1869473646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca8102320 0x7f7ca81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 -- 172.21.15.37:0/1869473646 >> 172.21.15.37:0/1869473646 conn(0x7f7ca80792d0 msgr2=0x7f7ca80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 -- 172.21.15.37:0/1869473646 shutdown_connections 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.822+0000 7f7caf586700 1 -- 172.21.15.37:0/1869473646 wait complete. 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 Processor -- start 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 -- start start 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca8102320 0x7f7ca819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca8108390 0x7f7ca81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8112530 con 0x7f7ca8102ce0 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81123b0 con 0x7f7ca8102320 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7caf586700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81126b0 con 0x7f7ca8108390 2024-04-26T18:22:32.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7cacb21700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7cacb21700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41744/0 (socket says 172.21.15.37:41744) 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7cacb21700 1 -- 172.21.15.37:0/2811689501 learned_addr learned my addr 172.21.15.37:0/2811689501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.823+0000 7f7cad322700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca8102320 0x7f7ca819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cadb23700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca8108390 0x7f7ca81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 -- 172.21.15.37:0/2811689501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca8108390 msgr2=0x7f7ca81987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca8108390 0x7f7ca81987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 -- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca8102320 msgr2=0x7f7ca819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ca8102320 0x7f7ca819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca8198fb0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7cacb21700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca819eb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800b530 tx=0x7f7c9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98015020 con 0x7f7ca8102ce0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca81992a0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c98011ce0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98010dd0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.824+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81a54b0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.825+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ca8063bb0 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.826+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c98018020 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7c9e7fc700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7c940a3cd0 0x7f7c940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7cad322700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7c940a3cd0 0x7f7c940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f7c980d0360 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7c9e7fc700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] conn(0x7f7c940ad910 0x7f7c940afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c98018ec0 con 0x7f7c940ad910 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7c980d0710 con 0x7f7ca8102ce0 2024-04-26T18:22:32.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7cad322700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7c940a3cd0 0x7f7c940a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4005950 tx=0x7f7ca400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:32.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.827+0000 7f7cadb23700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] conn(0x7f7c940ad910 0x7f7c940afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:32.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.828+0000 7f7cadb23700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] conn(0x7f7c940ad910 0x7f7c940afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:32.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.829+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== osd.4 v2:172.21.15.120:6808/4165015053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c98018ec0 con 0x7f7c940ad910 2024-04-26T18:22:32.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.853+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 --> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ca8064030 con 0x7f7c940ad910 2024-04-26T18:22:32.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.853+0000 7f7c9e7fc700 1 -- 172.21.15.37:0/2811689501 <== osd.4 v2:172.21.15.120:6808/4165015053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ca8064030 con 0x7f7c940ad910 2024-04-26T18:22:32.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] conn(0x7f7c940ad910 msgr2=0x7f7c940afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] conn(0x7f7c940ad910 0x7f7c940afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:32.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7c940a3cd0 msgr2=0x7f7c940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7c940a3cd0 0x7f7c940a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4005950 tx=0x7f7ca400a300 comp rx=0 tx=0).stop 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 msgr2=0x7f7ca819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 --2- 172.21.15.37:0/2811689501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ca8102ce0 0x7f7ca819eb90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800b530 tx=0x7f7c9800b840 comp rx=0 tx=0).stop 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7cadb23700 1 -- 172.21.15.37:0/2811689501 reap_dead start 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 shutdown_connections 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 >> 172.21.15.37:0/2811689501 conn(0x7f7ca80792d0 msgr2=0x7f7ca8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 shutdown_connections 2024-04-26T18:22:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:32.854+0000 7f7caf586700 1 -- 172.21.15.37:0/2811689501 wait complete. 2024-04-26T18:22:33.559 INFO:teuthology.orchestra.run.smithi037.stdout:55834574928 2024-04-26T18:22:33.559 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T18:22:33.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:33 smithi037 ceph-mon[29171]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:34.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:33 smithi120 ceph-mon[32244]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:34.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:33 smithi170 ceph-mon[32697]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:34.335 INFO:teuthology.orchestra.run.smithi037.stdout:163208757310 2024-04-26T18:22:34.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T18:22:34.818 INFO:teuthology.orchestra.run.smithi037.stdout:107374182469 2024-04-26T18:22:34.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T18:22:35.267 INFO:teuthology.orchestra.run.smithi037.stdout:309237645349 2024-04-26T18:22:35.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T18:22:35.783 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:35 smithi037 ceph-mon[29171]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.889+0000 7f7d9c856700 1 -- 172.21.15.37:0/4113539080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981094e0 msgr2=0x7f7d981098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.889+0000 7f7d9c856700 1 --2- 172.21.15.37:0/4113539080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981094e0 0x7f7d981098c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009990 tx=0x7f7d88009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 -- 172.21.15.37:0/4113539080 shutdown_connections 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 --2- 172.21.15.37:0/4113539080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d98103e60 0x7f7d981102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 --2- 172.21.15.37:0/4113539080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981094e0 0x7f7d981098c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 --2- 172.21.15.37:0/4113539080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d981034a0 0x7f7d98103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 -- 172.21.15.37:0/4113539080 >> 172.21.15.37:0/4113539080 conn(0x7f7d9807a590 msgr2=0x7f7d9807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 -- 172.21.15.37:0/4113539080 shutdown_connections 2024-04-26T18:22:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 -- 172.21.15.37:0/4113539080 wait complete. 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 Processor -- start 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.890+0000 7f7d9c856700 1 -- start start 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d981034a0 0x7f7d98075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 0x7f7d98075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d981094e0 0x7f7d980761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98113660 con 0x7f7d98103e60 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981134e0 con 0x7f7d981094e0 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9c856700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d981137e0 con 0x7f7d981034a0 2024-04-26T18:22:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 0x7f7d98075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 0x7f7d98075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40520/0 (socket says 172.21.15.37:40520) 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 -- 172.21.15.37:0/1751434855 learned_addr learned my addr 172.21.15.37:0/1751434855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d9659c700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d981034a0 0x7f7d98075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d96d9d700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d981094e0 0x7f7d980761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 -- 172.21.15.37:0/1751434855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d981034a0 msgr2=0x7f7d98075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d981034a0 0x7f7d98075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 -- 172.21.15.37:0/1751434855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d981094e0 msgr2=0x7f7d980761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7d981094e0 0x7f7d980761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981a80c0 con 0x7f7d98103e60 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d95d9b700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 0x7f7d98075c60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d980 tx=0x7f7d8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c017070 con 0x7f7d98103e60 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.891+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c005020 con 0x7f7d98103e60 2024-04-26T18:22:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.892+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d981a83b0 con 0x7f7d98103e60 2024-04-26T18:22:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.892+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00fa10 con 0x7f7d98103e60 2024-04-26T18:22:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.892+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a88a0 con 0x7f7d98103e60 2024-04-26T18:22:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.892+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d98063af0 con 0x7f7d98103e60 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.894+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d8c004a30 con 0x7f7d98103e60 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.894+0000 7f7d877fe700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7d800a3cd0 0x7f7d800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.894+0000 7f7d9659c700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7d800a3cd0 0x7f7d800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f7d8c0d09c0 con 0x7f7d98103e60 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d877fe700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] conn(0x7f7d800ad910 0x7f7d800afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d800b0480 con 0x7f7d800ad910 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7d8c0d0e10 con 0x7f7d98103e60 2024-04-26T18:22:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d96d9d700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] conn(0x7f7d800ad910 0x7f7d800afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d9659c700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7d800a3cd0 0x7f7d800a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b5c0 tx=0x7f7d88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.895+0000 7f7d96d9d700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] conn(0x7f7d800ad910 0x7f7d800afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.896+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== osd.6 v2:172.21.15.37:6818/1123986561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d800b0480 con 0x7f7d800ad910 2024-04-26T18:22:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 --> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d98063f70 con 0x7f7d800ad910 2024-04-26T18:22:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d877fe700 1 -- 172.21.15.37:0/1751434855 <== osd.6 v2:172.21.15.37:6818/1123986561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d98063f70 con 0x7f7d800ad910 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] conn(0x7f7d800ad910 msgr2=0x7f7d800afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] conn(0x7f7d800ad910 0x7f7d800afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7d800a3cd0 msgr2=0x7f7d800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7d800a3cd0 0x7f7d800a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b5c0 tx=0x7f7d88005fd0 comp rx=0 tx=0).stop 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 msgr2=0x7f7d98075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 --2- 172.21.15.37:0/1751434855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d98103e60 0x7f7d98075c60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00d980 tx=0x7f7d8c00dc90 comp rx=0 tx=0).stop 2024-04-26T18:22:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d96d9d700 1 -- 172.21.15.37:0/1751434855 reap_dead start 2024-04-26T18:22:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 shutdown_connections 2024-04-26T18:22:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.913+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 >> 172.21.15.37:0/1751434855 conn(0x7f7d9807a590 msgr2=0x7f7d98102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.914+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 shutdown_connections 2024-04-26T18:22:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:35.914+0000 7f7d9c856700 1 -- 172.21.15.37:0/1751434855 wait complete. 2024-04-26T18:22:36.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:35 smithi120 ceph-mon[32244]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:36.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:35 smithi170 ceph-mon[32697]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:36.640 INFO:teuthology.orchestra.run.smithi037.stdout:141733920836 2024-04-26T18:22:36.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T18:22:37.044 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.088 INFO:teuthology.orchestra.run.smithi037.stdout:197568495676 2024-04-26T18:22:37.088 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T18:22:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.759 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:37.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:37 smithi037 ceph-mon[29171]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:38.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:37 smithi120 ceph-mon[32244]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:38.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:37 smithi170 ceph-mon[32697]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:39.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:39 smithi037 ceph-mon[29171]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T18:22:40.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:39 smithi120 ceph-mon[32244]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T18:22:40.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:39 smithi170 ceph-mon[32697]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-26T18:22:41.780 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:41.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:41 smithi037 ceph-mon[29171]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T18:22:42.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:41 smithi120 ceph-mon[32244]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T18:22:42.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:41 smithi170 ceph-mon[32697]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 -- 172.21.15.37:0/3541166972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c1034c0 msgr2=0x7f0d2c103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 --2- 172.21.15.37:0/3541166972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c103940 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d200099c0 tx=0x7f0d20009cd0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 -- 172.21.15.37:0/3541166972 shutdown_connections 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 --2- 172.21.15.37:0/3541166972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c103e80 0x7f0d2c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 --2- 172.21.15.37:0/3541166972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c109560 0x7f0d2c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 --2- 172.21.15.37:0/3541166972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c103940 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.022+0000 7f0d30c14700 1 -- 172.21.15.37:0/3541166972 >> 172.21.15.37:0/3541166972 conn(0x7f0d2c07a5d0 msgr2=0x7f0d2c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.023+0000 7f0d30c14700 1 -- 172.21.15.37:0/3541166972 shutdown_connections 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.023+0000 7f0d30c14700 1 -- 172.21.15.37:0/3541166972 wait complete. 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.023+0000 7f0d30c14700 1 Processor -- start 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.023+0000 7f0d30c14700 1 -- start start 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c103e80 0x7f0d2c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c113660 con 0x7f0d2c109560 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1134e0 con 0x7f0d2c1034c0 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d30c14700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1137e0 con 0x7f0d2c103e80 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40542/0 (socket says 172.21.15.37:40542) 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 -- 172.21.15.37:0/1722386937 learned_addr learned my addr 172.21.15.37:0/1722386937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:43.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2a59c700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d29d9b700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c103e80 0x7f0d2c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 -- 172.21.15.37:0/1722386937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c103e80 msgr2=0x7f0d2c075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c103e80 0x7f0d2c075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 -- 172.21.15.37:0/1722386937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 msgr2=0x7f0d2c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 -- 172.21.15.37:0/1722386937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c1a7fe0 con 0x7f0d2c109560 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2a59c700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.024+0000 7f0d2ad9d700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007ee0 tx=0x7f0d2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.025+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24004bb0 con 0x7f0d2c109560 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.025+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c1a82d0 con 0x7f0d2c109560 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.025+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d24004d50 con 0x7f0d2c109560 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.025+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24005920 con 0x7f0d2c109560 2024-04-26T18:22:43.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.025+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c1a87e0 con 0x7f0d2c109560 2024-04-26T18:22:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.026+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2c10b9f0 con 0x7f0d2c109560 2024-04-26T18:22:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.028+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0d2401a020 con 0x7f0d2c109560 2024-04-26T18:22:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.028+0000 7f0d1b7fe700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 0x7f0d140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:43.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.028+0000 7f0d2a59c700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 0x7f0d140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.028+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f0d240d02a0 con 0x7f0d2c109560 2024-04-26T18:22:43.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.029+0000 7f0d2a59c700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 0x7f0d140a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009fd0 tx=0x7f0d1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:43.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.032+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d24010070 con 0x7f0d2c109560 2024-04-26T18:22:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.243+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0d2c104bd0 con 0x7f0d2c109560 2024-04-26T18:22:43.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.243+0000 7f0d1b7fe700 1 -- 172.21.15.37:0/1722386937 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d2c104bd0 con 0x7f0d2c109560 2024-04-26T18:22:43.243 INFO:teuthology.orchestra.run.smithi037.stdout:55834574931 2024-04-26T18:22:43.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 msgr2=0x7f0d140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 0x7f0d140a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009fd0 tx=0x7f0d1c009260 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 msgr2=0x7f0d2c0761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0d24007ee0 tx=0x7f0d2400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 shutdown_connections 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f0d140a3cd0 0x7f0d140a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d2c109560 0x7f0d2c0761a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d2c1034c0 0x7f0d2c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 --2- 172.21.15.37:0/1722386937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c103e80 0x7f0d2c075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 >> 172.21.15.37:0/1722386937 conn(0x7f0d2c07a5d0 msgr2=0x7f0d2c1021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 shutdown_connections 2024-04-26T18:22:43.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:43.245+0000 7f0d30c14700 1 -- 172.21.15.37:0/1722386937 wait complete. 2024-04-26T18:22:43.831 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:43 smithi037 ceph-mon[29171]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:43.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:43 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1722386937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:22:44.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:43 smithi120 ceph-mon[32244]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:44.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:43 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1722386937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:22:44.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:43 smithi170 ceph-mon[32697]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:44.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:43 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1722386937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:22:44.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 -- 172.21.15.37:0/2043934217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881034a0 msgr2=0x7fcf88103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:44.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/2043934217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881034a0 0x7fcf88103920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009990 tx=0x7fcf74009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 -- 172.21.15.37:0/2043934217 shutdown_connections 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/2043934217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf881102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/2043934217 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881094e0 0x7fcf881098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/2043934217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881034a0 0x7fcf88103920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 -- 172.21.15.37:0/2043934217 >> 172.21.15.37:0/2043934217 conn(0x7fcf8807a590 msgr2=0x7fcf8807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 -- 172.21.15.37:0/2043934217 shutdown_connections 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.669+0000 7fcf8eeae700 1 -- 172.21.15.37:0/2043934217 wait complete. 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 Processor -- start 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 -- start start 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881034a0 0x7fcf8819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf8819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88113660 con 0x7fcf881094e0 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881134e0 con 0x7fcf881034a0 2024-04-26T18:22:44.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8eeae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881137e0 con 0x7fcf88103e60 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8d44b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8d44b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40558/0 (socket says 172.21.15.37:40558) 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8d44b700 1 -- 172.21.15.37:0/676289842 learned_addr learned my addr 172.21.15.37:0/676289842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf8cc4a700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881034a0 0x7fcf8819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.670+0000 7fcf87fff700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf8819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 -- 172.21.15.37:0/676289842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 msgr2=0x7fcf8819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf8819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 -- 172.21.15.37:0/676289842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881034a0 msgr2=0x7fcf8819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881034a0 0x7fcf8819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 -- 172.21.15.37:0/676289842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8819a0e0 con 0x7fcf881094e0 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf87fff700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf8819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:44.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8d44b700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007d70 tx=0x7fcf8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80016020 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf8000ad40 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8819a3d0 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf800115e0 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.671+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881a6690 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.672+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8806a110 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.673+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf80019020 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.674+0000 7fcf85ffb700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 0x7fcf780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.674+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7fcf800d0400 con 0x7fcf881094e0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.674+0000 7fcf8cc4a700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 0x7fcf780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.674+0000 7fcf8cc4a700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 0x7fcf780a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf8819aac0 tx=0x7fcf7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:44.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.678+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf80020020 con 0x7fcf881094e0 2024-04-26T18:22:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.896+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcf8819aeb0 con 0x7fcf881094e0 2024-04-26T18:22:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.897+0000 7fcf85ffb700 1 -- 172.21.15.37:0/676289842 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf8009cd70 con 0x7fcf881094e0 2024-04-26T18:22:44.897 INFO:teuthology.orchestra.run.smithi037.stdout:163208757314 2024-04-26T18:22:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 msgr2=0x7fcf780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 0x7fcf780a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf8819aac0 tx=0x7fcf7c009190 comp rx=0 tx=0).stop 2024-04-26T18:22:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 msgr2=0x7fcf88199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf80007d70 tx=0x7fcf8000a340 comp rx=0 tx=0).stop 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 shutdown_connections 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fcf780a3d50 0x7fcf780a6210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf881094e0 0x7fcf88199980 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf881034a0 0x7fcf8819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 --2- 172.21.15.37:0/676289842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf88103e60 0x7fcf8819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 >> 172.21.15.37:0/676289842 conn(0x7fcf8807a590 msgr2=0x7fcf88102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 shutdown_connections 2024-04-26T18:22:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:44.899+0000 7fcf8eeae700 1 -- 172.21.15.37:0/676289842 wait complete. 2024-04-26T18:22:46.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:45 smithi120 ceph-mon[32244]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:46.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:45 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/676289842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:22:46.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:45 smithi037 ceph-mon[29171]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:46.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:45 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/676289842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:22:46.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:45 smithi170 ceph-mon[32697]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:46.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:45 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/676289842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:22:46.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757310 got 163208757314 for osd.5 2024-04-26T18:22:46.086 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:46.101 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:46.101 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:46.101 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:47 smithi037 ceph-mon[29171]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:48.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:47 smithi120 ceph-mon[32244]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:48.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:47 smithi170 ceph-mon[32697]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 -- 172.21.15.37:0/391384083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06b130 msgr2=0x7faa3c06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 --2- 172.21.15.37:0/391384083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06b130 0x7faa3c06b590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa38009990 tx=0x7faa38009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 -- 172.21.15.37:0/391384083 shutdown_connections 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 --2- 172.21.15.37:0/391384083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06bad0 0x7faa3c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 --2- 172.21.15.37:0/391384083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 --2- 172.21.15.37:0/391384083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06b130 0x7faa3c06b590 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.640+0000 7faa44112700 1 -- 172.21.15.37:0/391384083 >> 172.21.15.37:0/391384083 conn(0x7faa3c0792d0 msgr2=0x7faa3c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- 172.21.15.37:0/391384083 shutdown_connections 2024-04-26T18:22:48.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- 172.21.15.37:0/391384083 wait complete. 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 Processor -- start 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- start start 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06b130 0x7faa3c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c113660 con 0x7faa3c06bad0 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1134e0 con 0x7faa3c1094a0 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.641+0000 7faa44112700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c1137e0 con 0x7faa3c06b130 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46070/0 (socket says 172.21.15.37:46070) 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 -- 172.21.15.37:0/3347386091 learned_addr learned my addr 172.21.15.37:0/3347386091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:48.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa41eae700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06b130 0x7faa3c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa426af700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 -- 172.21.15.37:0/3347386091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06b130 msgr2=0x7faa3c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06b130 0x7faa3c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 -- 172.21.15.37:0/3347386091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 msgr2=0x7faa3c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 -- 172.21.15.37:0/3347386091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c19a180 con 0x7faa3c06bad0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa426af700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa416ad700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38026070 con 0x7faa3c06bad0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38003f20 con 0x7faa3c06bad0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3800fce0 con 0x7faa3c06bad0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c19a410 con 0x7faa3c06bad0 2024-04-26T18:22:48.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.642+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c1a6730 con 0x7faa3c06bad0 2024-04-26T18:22:48.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.643+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3c10ba10 con 0x7faa3c06bad0 2024-04-26T18:22:48.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.644+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faa38029070 con 0x7faa3c06bad0 2024-04-26T18:22:48.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.645+0000 7faa32ffd700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 0x7faa280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:48.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.645+0000 7faa41eae700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 0x7faa280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:48.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.645+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7faa380d0f00 con 0x7faa3c06bad0 2024-04-26T18:22:48.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.646+0000 7faa41eae700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 0x7faa280a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa3c19ab60 tx=0x7faa2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:48.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.650+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa3805dff0 con 0x7faa3c06bad0 2024-04-26T18:22:48.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.866+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faa3c19af50 con 0x7faa3c06bad0 2024-04-26T18:22:48.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.867+0000 7faa32ffd700 1 -- 172.21.15.37:0/3347386091 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa38013070 con 0x7faa3c06bad0 2024-04-26T18:22:48.866 INFO:teuthology.orchestra.run.smithi037.stdout:287762808878 2024-04-26T18:22:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 msgr2=0x7faa280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 0x7faa280a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa3c19ab60 tx=0x7faa2c009190 comp rx=0 tx=0).stop 2024-04-26T18:22:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 msgr2=0x7faa3c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa380057c0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 shutdown_connections 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faa280a3d50 0x7faa280a6210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa3c1094a0 0x7faa3c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3c06b130 0x7faa3c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 --2- 172.21.15.37:0/3347386091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa3c06bad0 0x7faa3c19fda0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 >> 172.21.15.37:0/3347386091 conn(0x7faa3c0792d0 msgr2=0x7faa3c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 shutdown_connections 2024-04-26T18:22:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:48.869+0000 7faa44112700 1 -- 172.21.15.37:0/3347386091 wait complete. 2024-04-26T18:22:49.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808874 got 287762808878 for osd.9 2024-04-26T18:22:49.712 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:49.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:49 smithi037 ceph-mon[29171]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:49.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:49 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3347386091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:22:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:49 smithi120 ceph-mon[32244]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:50.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:49 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3347386091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:22:50.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:49 smithi170 ceph-mon[32697]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:22:50.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:49 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3347386091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:22:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- 172.21.15.37:0/1509820356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 msgr2=0x7f80ac10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 --2- 172.21.15.37:0/1509820356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac10c1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80a400b0a0 tx=0x7f80a400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- 172.21.15.37:0/1509820356 shutdown_connections 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 --2- 172.21.15.37:0/1509820356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 0x7f80ac117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 --2- 172.21.15.37:0/1509820356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac10c1a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 --2- 172.21.15.37:0/1509820356 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 0x7f80ac114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- 172.21.15.37:0/1509820356 >> 172.21.15.37:0/1509820356 conn(0x7f80ac06e590 msgr2=0x7f80ac06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- 172.21.15.37:0/1509820356 shutdown_connections 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- 172.21.15.37:0/1509820356 wait complete. 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 Processor -- start 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.563+0000 7f80b3140700 1 -- start start 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 0x7f80ac115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 0x7f80ac1162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac11ad30 con 0x7f80ac10bdc0 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac11abb0 con 0x7f80ac10c6e0 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b3140700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac11aeb0 con 0x7f80ac115370 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46082/0 (socket says 172.21.15.37:46082) 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 -- 172.21.15.37:0/3427462350 learned_addr learned my addr 172.21.15.37:0/3427462350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80abfff700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 0x7f80ac115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b16dd700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 0x7f80ac1162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 -- 172.21.15.37:0/3427462350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 msgr2=0x7f80ac1162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 0x7f80ac1162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 -- 172.21.15.37:0/3427462350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 msgr2=0x7f80ac115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 0x7f80ac115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 -- 172.21.15.37:0/3427462350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac1b08d0 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80b0edc700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006070 tx=0x7f80a4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.564+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4021070 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.565+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a4003c00 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.565+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40047b0 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.565+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ac1b0b60 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.565+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ac1b1010 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.566+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ac06a110 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.568+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80a4007a00 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.568+0000 7f80a9ffb700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 0x7f80940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.569+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f80a400f020 con 0x7f80ac10bdc0 2024-04-26T18:22:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.569+0000 7f80abfff700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 0x7f80940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:50.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.569+0000 7f80abfff700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 0x7f80940a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80ac06edd0 tx=0x7f809c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:50.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.574+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80a402a020 con 0x7f80ac10bdc0 2024-04-26T18:22:50.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.787+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f80ac02e490 con 0x7f80ac10bdc0 2024-04-26T18:22:50.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.787+0000 7f80a9ffb700 1 -- 172.21.15.37:0/3427462350 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80a40bce20 con 0x7f80ac10bdc0 2024-04-26T18:22:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:34359738459 2024-04-26T18:22:50.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 msgr2=0x7f80940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 0x7f80940a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80ac06edd0 tx=0x7f809c00a250 comp rx=0 tx=0).stop 2024-04-26T18:22:50.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 msgr2=0x7f80ac115820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f80a4006070 tx=0x7f80a4004e40 comp rx=0 tx=0).stop 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 shutdown_connections 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f80940a3cd0 0x7f80940a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80ac115370 0x7f80ac1162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80ac10bdc0 0x7f80ac115820 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 --2- 172.21.15.37:0/3427462350 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80ac10c6e0 0x7f80ac115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 >> 172.21.15.37:0/3427462350 conn(0x7f80ac06e590 msgr2=0x7f80ac073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 shutdown_connections 2024-04-26T18:22:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:50.789+0000 7f80b3140700 1 -- 172.21.15.37:0/3427462350 wait complete. 2024-04-26T18:22:51.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 -- 172.21.15.37:0/1530082922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b901094f0 msgr2=0x7f1b901098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 --2- 172.21.15.37:0/1530082922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b901094f0 0x7f1b901098d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009990 tx=0x7f1b8c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 -- 172.21.15.37:0/1530082922 shutdown_connections 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 --2- 172.21.15.37:0/1530082922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b90103e90 0x7f1b901102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 --2- 172.21.15.37:0/1530082922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b901094f0 0x7f1b901098d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 --2- 172.21.15.37:0/1530082922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901034d0 0x7f1b90103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 -- 172.21.15.37:0/1530082922 >> 172.21.15.37:0/1530082922 conn(0x7f1b9007a500 msgr2=0x7f1b9007a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 -- 172.21.15.37:0/1530082922 shutdown_connections 2024-04-26T18:22:51.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.605+0000 7f1b97638700 1 -- 172.21.15.37:0/1530082922 wait complete. 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 Processor -- start 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 -- start start 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 0x7f1b901a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 0x7f1b9019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901135f0 con 0x7f1b90103e90 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90113470 con 0x7f1b901094f0 2024-04-26T18:22:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b97638700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90113770 con 0x7f1b901034d0 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46102/0 (socket says 172.21.15.37:46102) 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 -- 172.21.15.37:0/3091314561 learned_addr learned my addr 172.21.15.37:0/3091314561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b953d4700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 0x7f1b901a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b95bd5700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 0x7f1b9019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 -- 172.21.15.37:0/3091314561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 msgr2=0x7f1b901a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 0x7f1b901a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 -- 172.21.15.37:0/3091314561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 msgr2=0x7f1b9019bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.606+0000 7f1b94bd3700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 0x7f1b9019bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b94bd3700 1 -- 172.21.15.37:0/3091314561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9019c240 con 0x7f1b90103e90 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b953d4700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 0x7f1b901a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b95bd5700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 0x7f1b9019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:51.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b94bd3700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c920 tx=0x7f1b8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007b00 con 0x7f1b90103e90 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9019c530 con 0x7f1b90103e90 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b80004ce0 con 0x7f1b90103e90 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80005650 con 0x7f1b90103e90 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.607+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b901a87f0 con 0x7f1b90103e90 2024-04-26T18:22:51.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.608+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9006a110 con 0x7f1b90103e90 2024-04-26T18:22:51.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.609+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b80007ca0 con 0x7f1b90103e90 2024-04-26T18:22:51.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.610+0000 7f1b867fc700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 0x7f1b7c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:51.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.610+0000 7f1b953d4700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 0x7f1b7c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:51.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.610+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f1b800d0580 con 0x7f1b90103e90 2024-04-26T18:22:51.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.610+0000 7f1b953d4700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 0x7f1b7c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b5c0 tx=0x7f1b8c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:51.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.614+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b80014030 con 0x7f1b90103e90 2024-04-26T18:22:51.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.838+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1b9002e490 con 0x7f1b90103e90 2024-04-26T18:22:51.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.839+0000 7f1b867fc700 1 -- 172.21.15.37:0/3091314561 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1b8009cde0 con 0x7f1b90103e90 2024-04-26T18:22:51.839 INFO:teuthology.orchestra.run.smithi037.stdout:343597383717 2024-04-26T18:22:51.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 msgr2=0x7f1b7c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 0x7f1b7c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b5c0 tx=0x7f1b8c005a90 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 msgr2=0x7f1b901a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c920 tx=0x7f1b8000cce0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 shutdown_connections 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f1b7c0a3cd0 0x7f1b7c0a6190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b901094f0 0x7f1b9019bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b901034d0 0x7f1b901a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 --2- 172.21.15.37:0/3091314561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b90103e90 0x7f1b901a1e60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 >> 172.21.15.37:0/3091314561 conn(0x7f1b9007a500 msgr2=0x7f1b9010e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 shutdown_connections 2024-04-26T18:22:51.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.841+0000 7f1b97638700 1 -- 172.21.15.37:0/3091314561 wait complete. 2024-04-26T18:22:51.938 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:51 smithi037 ceph-mon[29171]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:51.938 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:51 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3427462350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:22:52.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:51 smithi120 ceph-mon[32244]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:52.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:51 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3427462350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:22:52.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:51 smithi170 ceph-mon[32697]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T18:22:52.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:51 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3427462350' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:22:53.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:52 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3091314561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:22:53.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:52 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2925186499' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:22:53.069 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:52 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3091314561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:22:53.070 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:52 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2925186499' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:22:53.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:52 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3091314561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:22:53.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:52 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2925186499' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:22:53.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:53 smithi037 ceph-mon[29171]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:54.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:53 smithi170 ceph-mon[32697]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:54.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:53 smithi120 ceph-mon[32244]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:22:54.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574931 for osd.1 2024-04-26T18:22:54.437 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:54.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 -- 172.21.15.37:0/2385249485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c103890 msgr2=0x7ff82c10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:54.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 --2- 172.21.15.37:0/2385249485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c103890 0x7ff82c10f2a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff82400b0d0 tx=0x7ff82400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 -- 172.21.15.37:0/2385249485 shutdown_connections 2024-04-26T18:22:54.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 --2- 172.21.15.37:0/2385249485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c103890 0x7ff82c10f2a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 --2- 172.21.15.37:0/2385249485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 0x7ff82c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 --2- 172.21.15.37:0/2385249485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c102ed0 0x7ff82c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 -- 172.21.15.37:0/2385249485 >> 172.21.15.37:0/2385249485 conn(0x7ff82c077640 msgr2=0x7ff82c077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 -- 172.21.15.37:0/2385249485 shutdown_connections 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.790+0000 7ff833134700 1 -- 172.21.15.37:0/2385249485 wait complete. 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 Processor -- start 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 -- start start 2024-04-26T18:22:54.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 0x7ff82c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 0x7ff82c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c112610 con 0x7ff82c102ed0 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c112490 con 0x7ff82c1085f0 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff833134700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c112790 con 0x7ff82c103890 2024-04-26T18:22:54.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46118/0 (socket says 172.21.15.37:46118) 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 -- 172.21.15.37:0/2925186499 learned_addr learned my addr 172.21.15.37:0/2925186499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff8316d1700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 0x7ff82c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff82bfff700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 0x7ff82c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 -- 172.21.15.37:0/2925186499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 msgr2=0x7ff82c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:54.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 0x7ff82c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 -- 172.21.15.37:0/2925186499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 msgr2=0x7ff82c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 0x7ff82c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff830ed0700 1 -- 172.21.15.37:0/2925186499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c19a040 con 0x7ff82c102ed0 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.791+0000 7ff82bfff700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 0x7ff82c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff830ed0700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c9b0 tx=0x7ff81400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814007b00 con 0x7ff82c102ed0 2024-04-26T18:22:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c19a330 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814007ca0 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814018660 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.792+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c1a65a0 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.793+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c06a110 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.795+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff81400f450 con 0x7ff82c102ed0 2024-04-26T18:22:54.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.795+0000 7ff829ffb700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 0x7ff8180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.796+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7ff8140d0810 con 0x7ff82c102ed0 2024-04-26T18:22:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.796+0000 7ff82bfff700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 0x7ff8180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.796+0000 7ff82bfff700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 0x7ff8180a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff82c077c40 tx=0x7ff81c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:51.799+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff814014030 con 0x7ff82c102ed0 2024-04-26T18:22:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.027+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff82c19adc0 con 0x7ff82c102ed0 2024-04-26T18:22:54.526 INFO:teuthology.orchestra.run.smithi037.stdout:253403070515 2024-04-26T18:22:54.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.027+0000 7ff829ffb700 1 -- 172.21.15.37:0/2925186499 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff81409d100 con 0x7ff82c102ed0 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.029+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 msgr2=0x7ff8180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.029+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 0x7ff8180a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff82c077c40 tx=0x7ff81c00b2c0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.029+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 msgr2=0x7ff82c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.029+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c9b0 tx=0x7ff81400ccc0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 shutdown_connections 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff8180a3cd0 0x7ff8180a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff82c1085f0 0x7ff82c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff82c102ed0 0x7ff82c19f6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 --2- 172.21.15.37:0/2925186499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff82c103890 0x7ff82c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 >> 172.21.15.37:0/2925186499 conn(0x7ff82c077640 msgr2=0x7ff82c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 shutdown_connections 2024-04-26T18:22:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:52.030+0000 7ff833134700 1 -- 172.21.15.37:0/2925186499 wait complete. 2024-04-26T18:22:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:55.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:54 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2863043769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:22:55.088 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:54 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2863043769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:22:55.119 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070510 got 253403070515 for osd.8 2024-04-26T18:22:55.120 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:55.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:54 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2863043769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:22:56.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:55 smithi037 ceph-mon[29171]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:56.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:55 smithi120 ceph-mon[32244]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:56.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:55 smithi170 ceph-mon[32697]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:22:56.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738459 for osd.0 2024-04-26T18:22:56.405 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 -- 172.21.15.37:0/1070657885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 msgr2=0x7f2770102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 --2- 172.21.15.37:0/1070657885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f2770102d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f27640099c0 tx=0x7f2764009cd0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 -- 172.21.15.37:0/1070657885 shutdown_connections 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 --2- 172.21.15.37:0/1070657885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27701076b0 0x7f277010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 --2- 172.21.15.37:0/1070657885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f2770102d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 --2- 172.21.15.37:0/1070657885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2770103310 0x7f2770106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 -- 172.21.15.37:0/1070657885 >> 172.21.15.37:0/1070657885 conn(0x7f27700792d0 msgr2=0x7f27700796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 -- 172.21.15.37:0/1070657885 shutdown_connections 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.625+0000 7f27770f4700 1 -- 172.21.15.37:0/1070657885 wait complete. 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 Processor -- start 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 -- start start 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 0x7f277019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27701076b0 0x7f2770199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2770112560 con 0x7f2770102960 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27701123e0 con 0x7f27701076b0 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f27770f4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27701126e0 con 0x7f2770103310 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2774e90700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2774e90700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46142/0 (socket says 172.21.15.37:46142) 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2774e90700 1 -- 172.21.15.37:0/2863043769 learned_addr learned my addr 172.21.15.37:0/2863043769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2775691700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27701076b0 0x7f2770199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f276ffff700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 0x7f277019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2774e90700 1 -- 172.21.15.37:0/2863043769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 msgr2=0x7f277019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.626+0000 7f2774e90700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 0x7f277019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f2774e90700 1 -- 172.21.15.37:0/2863043769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27701076b0 msgr2=0x7f2770199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f2774e90700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27701076b0 0x7f2770199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f2774e90700 1 -- 172.21.15.37:0/2863043769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277019a160 con 0x7f2770102960 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f276ffff700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 0x7f277019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f2774e90700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2764009990 tx=0x7f27640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764027070 con 0x7f2770102960 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27640040a0 con 0x7f2770102960 2024-04-26T18:22:56.754 INFO:teuthology.orchestra.run.smithi037.stdout:77309411410 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277019a3f0 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764022e60 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.627+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27701a6710 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.628+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2770103a60 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.630+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f276402a070 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.631+0000 7f276dffb700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 0x7f27600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.631+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8269+0+0 (secure 0 0 0) 0x7f27640d1550 con 0x7f2770102960 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.631+0000 7f276ffff700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 0x7f27600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.631+0000 7f276ffff700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 0x7f27600a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f275c005fd0 tx=0x7f275c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:56.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.635+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2764013070 con 0x7f2770102960 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.849+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f277002e490 con 0x7f2770102960 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.850+0000 7f276dffb700 1 -- 172.21.15.37:0/2863043769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f276409de40 con 0x7f2770102960 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 msgr2=0x7f27600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 0x7f27600a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f275c005fd0 tx=0x7f275c005dc0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 msgr2=0x7f277019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2764009990 tx=0x7f27640047b0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 shutdown_connections 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f27600a3d50 0x7f27600a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27701076b0 0x7f2770199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2770102960 0x7f277019f840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 --2- 172.21.15.37:0/2863043769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2770103310 0x7f277019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 >> 172.21.15.37:0/2863043769 conn(0x7f27700792d0 msgr2=0x7f277006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 shutdown_connections 2024-04-26T18:22:56.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:53.852+0000 7f27770f4700 1 -- 172.21.15.37:0/2863043769 wait complete. 2024-04-26T18:22:57.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411410 for osd.2 2024-04-26T18:22:57.557 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 -- 172.21.15.37:0/1142612091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd10103e80 msgr2=0x7fbd10110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1142612091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd10103e80 0x7fbd10110330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800b0d0 tx=0x7fbd0800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 -- 172.21.15.37:0/1142612091 shutdown_connections 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1142612091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd10103e80 0x7fbd10110330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1142612091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10109560 0x7fbd10109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1142612091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd101034c0 0x7fbd10103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 -- 172.21.15.37:0/1142612091 >> 172.21.15.37:0/1142612091 conn(0x7fbd1007a5d0 msgr2=0x7fbd1007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 -- 172.21.15.37:0/1142612091 shutdown_connections 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.021+0000 7fbd1512c700 1 -- 172.21.15.37:0/1142612091 wait complete. 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 Processor -- start 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 -- start start 2024-04-26T18:22:58.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10103e80 0x7fbd1019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd10109560 0x7fbd10199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10113640 con 0x7fbd101034c0 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd101134c0 con 0x7fbd10103e80 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.022+0000 7fbd1512c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd101137c0 con 0x7fbd10109560 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36848/0 (socket says 172.21.15.37:36848) 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 -- 172.21.15.37:0/1337978664 learned_addr learned my addr 172.21.15.37:0/1337978664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0e59c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10103e80 0x7fbd1019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0f59e700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd10109560 0x7fbd10199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 -- 172.21.15.37:0/1337978664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd10109560 msgr2=0x7fbd10199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd10109560 0x7fbd10199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 -- 172.21.15.37:0/1337978664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10103e80 msgr2=0x7fbd1019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10103e80 0x7fbd1019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 -- 172.21.15.37:0/1337978664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1019a020 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd0ed9d700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800c9b0 tx=0x7fbcf800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8007b00 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf8007ca0 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1019a310 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8018660 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.023+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101a64d0 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.024+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd10104bd0 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.025+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbcf800f450 con 0x7fbd101034c0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.026+0000 7fbd07fff700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 0x7fbcfc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.026+0000 7fbd0e59c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 0x7fbcfc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.026+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fbcf80d0440 con 0x7fbd101034c0 2024-04-26T18:22:58.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.027+0000 7fbd0e59c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 0x7fbcfc0a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd1007aad0 tx=0x7fbd0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:58.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.031+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcf8014030 con 0x7fbd101034c0 2024-04-26T18:22:58.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:57 smithi037 ceph-mon[29171]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:58.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:57 smithi037 ceph-mon[29171]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-26T18:22:58.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:57 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T18:22:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.245+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbd1002e490 con 0x7fbd101034c0 2024-04-26T18:22:58.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.245+0000 7fbd07fff700 1 -- 172.21.15.37:0/1337978664 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbcf809cdf0 con 0x7fbd101034c0 2024-04-26T18:22:58.245 INFO:teuthology.orchestra.run.smithi037.stdout:219043332153 2024-04-26T18:22:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 msgr2=0x7fbcfc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 0x7fbcfc0a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd1007aad0 tx=0x7fbd0000b410 comp rx=0 tx=0).stop 2024-04-26T18:22:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 msgr2=0x7fbd1019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbcf800c9b0 tx=0x7fbcf800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 shutdown_connections 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbcfc0a3c40 0x7fbcfc0a6100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd10109560 0x7fbd10199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd101034c0 0x7fbd1019f670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 --2- 172.21.15.37:0/1337978664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd10103e80 0x7fbd1019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 >> 172.21.15.37:0/1337978664 conn(0x7fbd1007a5d0 msgr2=0x7fbd10101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 shutdown_connections 2024-04-26T18:22:58.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.247+0000 7fbd1512c700 1 -- 172.21.15.37:0/1337978664 wait complete. 2024-04-26T18:22:58.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:57 smithi120 ceph-mon[32244]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:58.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:57 smithi120 ceph-mon[32244]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-26T18:22:58.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:57 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T18:22:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:57 smithi170 ceph-mon[32697]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:22:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:57 smithi170 ceph-mon[32697]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-26T18:22:58.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:57 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-04-26T18:22:58.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383712 got 343597383717 for osd.11 2024-04-26T18:22:58.421 DEBUG:teuthology.parallel:result is None 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 -- 172.21.15.37:0/3661714734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 msgr2=0x7fe72810f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 --2- 172.21.15.37:0/3661714734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe72810f210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b0d0 tx=0x7fe72000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 -- 172.21.15.37:0/3661714734 shutdown_connections 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 --2- 172.21.15.37:0/3661714734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe72810f210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 --2- 172.21.15.37:0/3661714734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 --2- 172.21.15.37:0/3661714734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 -- 172.21.15.37:0/3661714734 >> 172.21.15.37:0/3661714734 conn(0x7fe7280792d0 msgr2=0x7fe7280796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 -- 172.21.15.37:0/3661714734 shutdown_connections 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.618+0000 7fe72ee63700 1 -- 172.21.15.37:0/3661714734 wait complete. 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 Processor -- start 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 -- start start 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe728112500 con 0x7fe728107640 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe728112380 con 0x7fe728102f60 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72ee63700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe728112680 con 0x7fe7281025b0 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36868/0 (socket says 172.21.15.37:36868) 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 -- 172.21.15.37:0/602766952 learned_addr learned my addr 172.21.15.37:0/602766952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:22:58.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe727fff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72cbff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 -- 172.21.15.37:0/602766952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 msgr2=0x7fe728104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 -- 172.21.15.37:0/602766952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 msgr2=0x7fe728104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72d400700 1 -- 172.21.15.37:0/602766952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7281a8070 con 0x7fe728107640 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe727fff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.619+0000 7fe72cbff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe72d400700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe720009fd0 tx=0x7fe720007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72001f070 con 0x7fe728107640 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe720003c00 con 0x7fe728107640 2024-04-26T18:22:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72001b8e0 con 0x7fe728107640 2024-04-26T18:22:58.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7281a8300 con 0x7fe728107640 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.620+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7281a8700 con 0x7fe728107640 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.621+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72806a110 con 0x7fe728107640 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.622+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe720022070 con 0x7fe728107640 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.623+0000 7fe725ffb700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 0x7fe7180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.623+0000 7fe72cbff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 0x7fe7180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.623+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe7200d16d0 con 0x7fe728107640 2024-04-26T18:22:58.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.623+0000 7fe72cbff700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 0x7fe7180a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005fd0 tx=0x7fe71c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:58.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.628+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe72009e0b0 con 0x7fe728107640 2024-04-26T18:22:58.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.836+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe72802e650 con 0x7fe728107640 2024-04-26T18:22:58.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.837+0000 7fe725ffb700 1 -- 172.21.15.37:0/602766952 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe72009d9b0 con 0x7fe728107640 2024-04-26T18:22:58.837 INFO:teuthology.orchestra.run.smithi037.stdout:309237645355 2024-04-26T18:22:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 msgr2=0x7fe7180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 0x7fe7180a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005fd0 tx=0x7fe71c005dc0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 msgr2=0x7fe728104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe720009fd0 tx=0x7fe720007ad0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 shutdown_connections 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fe7180a3c40 0x7fe7180a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe728107640 0x7fe728104a80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7281025b0 0x7fe728104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 --2- 172.21.15.37:0/602766952 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe728102f60 0x7fe728104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 >> 172.21.15.37:0/602766952 conn(0x7fe7280792d0 msgr2=0x7fe72806beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 shutdown_connections 2024-04-26T18:22:58.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:22:58.839+0000 7fe72ee63700 1 -- 172.21.15.37:0/602766952 wait complete. 2024-04-26T18:22:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:59 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T18:22:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:59 smithi037 ceph-mon[29171]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:22:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:59 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1337978664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:22:59.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:22:59 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/602766952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:22:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:59 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T18:22:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:59 smithi170 ceph-mon[32697]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:22:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:59 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1337978664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:22:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:22:59 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/602766952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:22:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:59 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-04-26T18:22:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:59 smithi120 ceph-mon[32244]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:22:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:59 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1337978664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:22:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:22:59 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/602766952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:22:59.854 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645349 got 309237645355 for osd.10 2024-04-26T18:22:59.854 DEBUG:teuthology.parallel:result is None 2024-04-26T18:23:00.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:00 smithi037 ceph-mon[29171]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T18:23:00.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:00 smithi037 ceph-mon[29171]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:23:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:00 smithi170 ceph-mon[32697]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T18:23:00.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:00 smithi170 ceph-mon[32697]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:23:00.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:00 smithi120 ceph-mon[32244]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T18:23:00.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:00 smithi120 ceph-mon[32244]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 -- 172.21.15.37:0/2606082887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010bdc0 msgr2=0x7f60f010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 --2- 172.21.15.37:0/2606082887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010bdc0 0x7f60f010c1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009990 tx=0x7f60e0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 -- 172.21.15.37:0/2606082887 shutdown_connections 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 --2- 172.21.15.37:0/2606082887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f0115370 0x7f60f0117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 --2- 172.21.15.37:0/2606082887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010bdc0 0x7f60f010c1a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 --2- 172.21.15.37:0/2606082887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f010c6e0 0x7f60f0114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 -- 172.21.15.37:0/2606082887 >> 172.21.15.37:0/2606082887 conn(0x7f60f006e590 msgr2=0x7f60f006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 -- 172.21.15.37:0/2606082887 shutdown_connections 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.426+0000 7f60f5319700 1 -- 172.21.15.37:0/2606082887 wait complete. 2024-04-26T18:23:01.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 Processor -- start 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 -- start start 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f010bdc0 0x7f60f0072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f0115370 0x7f60f0072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60f011ad50 con 0x7f60f010c6e0 2024-04-26T18:23:01.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60f011abd0 con 0x7f60f010bdc0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.427+0000 7f60f5319700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f60f011aed0 con 0x7f60f0115370 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36886/0 (socket says 172.21.15.37:36886) 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 -- 172.21.15.37:0/4219731149 learned_addr learned my addr 172.21.15.37:0/4219731149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60eeffd700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f010bdc0 0x7f60f0072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ef7fe700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f0115370 0x7f60f0072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 -- 172.21.15.37:0/4219731149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f0115370 msgr2=0x7f60f0072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f0115370 0x7f60f0072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 -- 172.21.15.37:0/4219731149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f010bdc0 msgr2=0x7f60f0072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f010bdc0 0x7f60f0072090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 -- 172.21.15.37:0/4219731149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f01b08d0 con 0x7f60f010c6e0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60ee7fc700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c900 tx=0x7f60e400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40078d0 con 0x7f60f010c6e0 2024-04-26T18:23:01.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f01b0bc0 con 0x7f60f010c6e0 2024-04-26T18:23:01.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.428+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e400f450 con 0x7f60f010c6e0 2024-04-26T18:23:01.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.429+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4018640 con 0x7f60f010c6e0 2024-04-26T18:23:01.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.429+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01b1150 con 0x7f60f010c6e0 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.430+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f006a110 con 0x7f60f010c6e0 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.431+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f60e400f600 con 0x7f60f010c6e0 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.431+0000 7f60d7fff700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 0x7f60d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.431+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f60e40d02c0 con 0x7f60f010c6e0 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.432+0000 7f60eeffd700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 0x7f60d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.432+0000 7f60eeffd700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 0x7f60d80a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60e00098e0 tx=0x7f60e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.437+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60e4014030 con 0x7f60f010c6e0 2024-04-26T18:23:01.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332148 got 219043332153 for osd.7 2024-04-26T18:23:01.568 DEBUG:teuthology.parallel:result is None 2024-04-26T18:23:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.650+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f60f002e490 con 0x7f60f010c6e0 2024-04-26T18:23:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.650+0000 7f60d7fff700 1 -- 172.21.15.37:0/4219731149 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f60e409cc70 con 0x7f60f010c6e0 2024-04-26T18:23:01.650 INFO:teuthology.orchestra.run.smithi037.stdout:107374182478 2024-04-26T18:23:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 msgr2=0x7f60d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 0x7f60d80a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f60e00098e0 tx=0x7f60e0005960 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 msgr2=0x7f60f00725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c900 tx=0x7f60e400cc10 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 shutdown_connections 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f60d80a3c40 0x7f60d80a6100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60f0115370 0x7f60f0072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60f010bdc0 0x7f60f0072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 --2- 172.21.15.37:0/4219731149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f010c6e0 0x7f60f00725d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 >> 172.21.15.37:0/4219731149 conn(0x7f60f006e590 msgr2=0x7f60f010b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 shutdown_connections 2024-04-26T18:23:01.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:01.652+0000 7f60f5319700 1 -- 172.21.15.37:0/4219731149 wait complete. 2024-04-26T18:23:02.129 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:02 smithi037 ceph-mon[29171]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:02.129 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:02 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4219731149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:23:02.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:02 smithi170 ceph-mon[32697]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:02.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:02 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4219731149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:23:02.425 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182469 got 107374182478 for osd.3 2024-04-26T18:23:02.425 DEBUG:teuthology.parallel:result is None 2024-04-26T18:23:02.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:02 smithi120 ceph-mon[32244]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:02.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:02 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4219731149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 -- 172.21.15.37:0/3921852141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103e30 msgr2=0x7f87a8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 --2- 172.21.15.37:0/3921852141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103e30 0x7f87a8110290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b0a0 tx=0x7f87a000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 -- 172.21.15.37:0/3921852141 shutdown_connections 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 --2- 172.21.15.37:0/3921852141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103e30 0x7f87a8110290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 --2- 172.21.15.37:0/3921852141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 0x7f87a81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 --2- 172.21.15.37:0/3921852141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103470 0x7f87a81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 -- 172.21.15.37:0/3921852141 >> 172.21.15.37:0/3921852141 conn(0x7f87a807a580 msgr2=0x7f87a807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 -- 172.21.15.37:0/3921852141 shutdown_connections 2024-04-26T18:23:02.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.636+0000 7f87ae22a700 1 -- 172.21.15.37:0/3921852141 wait complete. 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 Processor -- start 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 -- start start 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103e30 0x7f87a819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 0x7f87a8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81135c0 con 0x7f87a8103470 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87a8113440 con 0x7f87a8103e30 2024-04-26T18:23:02.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87ae22a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f87a8113740 con 0x7f87a81094c0 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.637+0000 7f87a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36902/0 (socket says 172.21.15.37:36902) 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 -- 172.21.15.37:0/252622437 learned_addr learned my addr 172.21.15.37:0/252622437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f879effd700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103e30 0x7f87a819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a7fff700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 0x7f87a8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 -- 172.21.15.37:0/252622437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 msgr2=0x7f87a8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 0x7f87a8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 -- 172.21.15.37:0/252622437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103e30 msgr2=0x7f87a819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103e30 0x7f87a819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 -- 172.21.15.37:0/252622437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a819a020 con 0x7f87a8103470 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a77fe700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f879800d950 tx=0x7f879800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798016070 con 0x7f87a8103470 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a819a310 con 0x7f87a8103470 2024-04-26T18:23:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879800fd40 con 0x7f87a8103470 2024-04-26T18:23:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879800e3b0 con 0x7f87a8103470 2024-04-26T18:23:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.638+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a81a6520 con 0x7f87a8103470 2024-04-26T18:23:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.639+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a8104b60 con 0x7f87a8103470 2024-04-26T18:23:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.640+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8798019070 con 0x7f87a8103470 2024-04-26T18:23:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.641+0000 7f87a57fa700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 0x7f87940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.641+0000 7f879effd700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 0x7f87940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.641+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f87980cfec0 con 0x7f87a8103470 2024-04-26T18:23:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.641+0000 7f879effd700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 0x7f87940a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87a807aaf0 tx=0x7f879000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:02.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.645+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f879809c810 con 0x7f87a8103470 2024-04-26T18:23:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.859+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f87a802e490 con 0x7f87a8103470 2024-04-26T18:23:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.860+0000 7f87a57fa700 1 -- 172.21.15.37:0/252622437 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f879809c110 con 0x7f87a8103470 2024-04-26T18:23:02.860 INFO:teuthology.orchestra.run.smithi037.stdout:197568495680 2024-04-26T18:23:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 msgr2=0x7f87940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 0x7f87940a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87a807aaf0 tx=0x7f879000a380 comp rx=0 tx=0).stop 2024-04-26T18:23:02.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 msgr2=0x7f87a819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f879800d950 tx=0x7f879800dc60 comp rx=0 tx=0).stop 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 shutdown_connections 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f87940a3c40 0x7f87940a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a81094c0 0x7f87a8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87a8103470 0x7f87a819f6c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 --2- 172.21.15.37:0/252622437 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87a8103e30 0x7f87a819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 >> 172.21.15.37:0/252622437 conn(0x7f87a807a580 msgr2=0x7f87a8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 shutdown_connections 2024-04-26T18:23:02.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:02.862+0000 7f87ae22a700 1 -- 172.21.15.37:0/252622437 wait complete. 2024-04-26T18:23:03.109 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:03 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/252622437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:23:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:03 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/252622437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:23:03.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:03 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/252622437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:23:03.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495676 got 197568495680 for osd.6 2024-04-26T18:23:03.731 DEBUG:teuthology.parallel:result is None 2024-04-26T18:23:04.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:04 smithi037 ceph-mon[29171]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:04.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3303502242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1025b0 msgr2=0x7f4fec102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:04.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3303502242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1025b0 0x7f4fec102990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00099c0 tx=0x7f4fe0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3303502242 shutdown_connections 2024-04-26T18:23:04.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3303502242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec107640 0x7f4fec10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3303502242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1025b0 0x7f4fec102990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3303502242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 0x7f4fec106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3303502242 >> 172.21.15.37:0/3303502242 conn(0x7f4fec0792d0 msgr2=0x7f4fec0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3303502242 shutdown_connections 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3303502242 wait complete. 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.304+0000 7f4ff2c3f700 1 Processor -- start 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 -- start start 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec1025b0 0x7f4fec19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 0x7f4fec19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1124d0 con 0x7f4fec107640 2024-04-26T18:23:04.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec112350 con 0x7f4fec1025b0 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff2c3f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec112650 con 0x7f4fec102f60 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36916/0 (socket says 172.21.15.37:36916) 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 -- 172.21.15.37:0/3970137799 learned_addr learned my addr 172.21.15.37:0/3970137799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff09db700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec1025b0 0x7f4fec19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4febfff700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 0x7f4fec19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 -- 172.21.15.37:0/3970137799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 msgr2=0x7f4fec19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 0x7f4fec19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 -- 172.21.15.37:0/3970137799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec1025b0 msgr2=0x7f4fec19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec1025b0 0x7f4fec19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 -- 172.21.15.37:0/3970137799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec1a5ef0 con 0x7f4fec107640 2024-04-26T18:23:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.305+0000 7f4ff11dc700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400ada0 tx=0x7f4fe400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.306+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4017020 con 0x7f4fec107640 2024-04-26T18:23:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.306+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec1a6180 con 0x7f4fec107640 2024-04-26T18:23:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.306+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe400cd80 con 0x7f4fec107640 2024-04-26T18:23:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.306+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4005260 con 0x7f4fec107640 2024-04-26T18:23:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.306+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a6690 con 0x7f4fec107640 2024-04-26T18:23:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.307+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec06a110 con 0x7f4fec107640 2024-04-26T18:23:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.308+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4fe400c8d0 con 0x7f4fec107640 2024-04-26T18:23:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.309+0000 7f4fe9ffb700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 0x7f4fdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.309+0000 7f4ff09db700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 0x7f4fdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.309+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f4fe40cfab0 con 0x7f4fec107640 2024-04-26T18:23:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.309+0000 7f4ff09db700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 0x7f4fdc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009990 tx=0x7f4fe0009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.313+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fe4021020 con 0x7f4fec107640 2024-04-26T18:23:04.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:04 smithi170 ceph-mon[32697]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:04 smithi120 ceph-mon[32244]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-26T18:23:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.521+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4fec02e490 con 0x7f4fec107640 2024-04-26T18:23:04.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.521+0000 7f4fe9ffb700 1 -- 172.21.15.37:0/3970137799 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4fe409c4f0 con 0x7f4fec107640 2024-04-26T18:23:04.521 INFO:teuthology.orchestra.run.smithi037.stdout:141733920842 2024-04-26T18:23:04.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 msgr2=0x7f4fdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 0x7f4fdc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009990 tx=0x7f4fe0009650 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 msgr2=0x7f4fec1a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400ada0 tx=0x7f4fe400c330 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 shutdown_connections 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f4fdc0a3c40 0x7f4fdc0a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec107640 0x7f4fec1a59b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec1025b0 0x7f4fec19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 --2- 172.21.15.37:0/3970137799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec102f60 0x7f4fec19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 >> 172.21.15.37:0/3970137799 conn(0x7f4fec0792d0 msgr2=0x7f4fec1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 shutdown_connections 2024-04-26T18:23:04.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:04.523+0000 7f4ff2c3f700 1 -- 172.21.15.37:0/3970137799 wait complete. 2024-04-26T18:23:05.095 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:05 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3970137799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:23:05.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920842 for osd.4 2024-04-26T18:23:05.126 DEBUG:teuthology.parallel:result is None 2024-04-26T18:23:05.126 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:23:05.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:23:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:05 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3970137799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:23:05.376 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:05 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3970137799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:23:06.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:06 smithi037 ceph-mon[29171]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 5 op/s 2024-04-26T18:23:06.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:06 smithi170 ceph-mon[32697]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 5 op/s 2024-04-26T18:23:06.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:06 smithi120 ceph-mon[32244]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 5 op/s 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 -- 172.21.15.37:0/3214258917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 msgr2=0x7efeb010f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 --2- 172.21.15.37:0/3214258917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb010f1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efea800b0a0 tx=0x7efea800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 -- 172.21.15.37:0/3214258917 shutdown_connections 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 --2- 172.21.15.37:0/3214258917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb010f1c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 --2- 172.21.15.37:0/3214258917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0108380 0x7efeb0108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 --2- 172.21.15.37:0/3214258917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0102310 0x7efeb0102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 -- 172.21.15.37:0/3214258917 >> 172.21.15.37:0/3214258917 conn(0x7efeb0079290 msgr2=0x7efeb00796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 -- 172.21.15.37:0/3214258917 shutdown_connections 2024-04-26T18:23:07.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.061+0000 7efeb754d700 1 -- 172.21.15.37:0/3214258917 wait complete. 2024-04-26T18:23:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.062+0000 7efeb754d700 1 Processor -- start 2024-04-26T18:23:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.062+0000 7efeb754d700 1 -- start start 2024-04-26T18:23:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.062+0000 7efeb754d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0102310 0x7efeb019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.062+0000 7efeb754d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb754d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0108380 0x7efeb0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb754d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efeb01124d0 con 0x7efeb0102cd0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb754d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112350 con 0x7efeb0102310 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb754d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0112650 con 0x7efeb0108380 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49924/0 (socket says 172.21.15.37:49924) 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 -- 172.21.15.37:0/2858789507 learned_addr learned my addr 172.21.15.37:0/2858789507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb5aea700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0108380 0x7efeb0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb52e9700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0102310 0x7efeb019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 -- 172.21.15.37:0/2858789507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0108380 msgr2=0x7efeb0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0108380 0x7efeb0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 -- 172.21.15.37:0/2858789507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0102310 msgr2=0x7efeb019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0102310 0x7efeb019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 -- 172.21.15.37:0/2858789507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb0199030 con 0x7efeb0102cd0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efeb4ae8700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efea000d980 tx=0x7efea000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0017070 con 0x7efeb0102cd0 2024-04-26T18:23:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.063+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea0005020 con 0x7efeb0102cd0 2024-04-26T18:23:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.064+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb0199320 con 0x7efeb0102cd0 2024-04-26T18:23:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.064+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea000fa10 con 0x7efeb0102cd0 2024-04-26T18:23:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.064+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb006b990 con 0x7efeb0102cd0 2024-04-26T18:23:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.065+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0103a30 con 0x7efeb0102cd0 2024-04-26T18:23:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.066+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efea0004a30 con 0x7efeb0102cd0 2024-04-26T18:23:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.066+0000 7efea67fc700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 0x7efe9c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.066+0000 7efeb52e9700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 0x7efe9c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.066+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7efea00d0250 con 0x7efeb0102cd0 2024-04-26T18:23:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.067+0000 7efeb52e9700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 0x7efe9c0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efeac009be0 tx=0x7efeac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:07.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.071+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efea0020020 con 0x7efeb0102cd0 2024-04-26T18:23:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.274+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efeb006bbf0 con 0x7efe9c0a3cc0 2024-04-26T18:23:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.281+0000 7efea67fc700 1 -- 172.21.15.37:0/2858789507 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258852 (secure 0 0 0) 0x7efeb006bbf0 con 0x7efe9c0a3cc0 2024-04-26T18:23:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 msgr2=0x7efe9c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 0x7efe9c0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efeac009be0 tx=0x7efeac009260 comp rx=0 tx=0).stop 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 msgr2=0x7efeb019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efea000d980 tx=0x7efea000dc90 comp rx=0 tx=0).stop 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 shutdown_connections 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7efe9c0a3cc0 0x7efe9c0a6180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb0108380 0x7efeb0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.284+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb0102310 0x7efeb019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.285+0000 7efeb754d700 1 --2- 172.21.15.37:0/2858789507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeb0102cd0 0x7efeb019ebc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.285+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 >> 172.21.15.37:0/2858789507 conn(0x7efeb0079290 msgr2=0x7efeb0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.285+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 shutdown_connections 2024-04-26T18:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:07.285+0000 7efeb754d700 1 -- 172.21.15.37:0/2858789507 wait complete. 2024-04-26T18:23:07.288 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:23:08.058 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-04-26T18:23:06.702087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":756,"num_read_kb":498,"num_write":354,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182012,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384116,"kb_used_data":14076,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557708,"statfs":{"total":1151940427776,"available":1151547092992,"internally_reserved":0,"allocated":14413824,"data_stored":7923972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.698347"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012366+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:22:59.012366+0000","last_peered":"2024-04-26T18:22:59.012366+0000","last_clean":"2024-04-26T18:22:59.012366+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:22:59.012366+0000","last_undegraded":"2024-04-26T18:22:59.012366+0000","last_fullsized":"2024-04-26T18:22:59.012366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014261+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:22:59.014261+0000","last_peered":"2024-04-26T18:22:59.014261+0000","last_clean":"2024-04-26T18:22:59.014261+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:22:59.014261+0000","last_undegraded":"2024-04-26T18:22:59.014261+0000","last_fullsized":"2024-04-26T18:22:59.014261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014249+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:22:59.014249+0000","last_peered":"2024-04-26T18:22:59.014249+0000","last_clean":"2024-04-26T18:22:59.014249+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:22:59.014249+0000","last_undegraded":"2024-04-26T18:22:59.014249+0000","last_fullsized":"2024-04-26T18:22:59.014249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014594+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:22:59.014594+0000","last_peered":"2024-04-26T18:22:59.014594+0000","last_clean":"2024-04-26T18:22:59.014594+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:22:59.014594+0000","last_undegraded":"2024-04-26T18:22:59.014594+0000","last_fullsized":"2024-04-26T18:22:59.014594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010368+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:22:59.010368+0000","last_peered":"2024-04-26T18:22:59.010368+0000","last_clean":"2024-04-26T18:22:59.010368+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:22:59.010368+0000","last_undegraded":"2024-04-26T18:22:59.010368+0000","last_fullsized":"2024-04-26T18:22:59.010368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011004+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:22:59.011004+0000","last_peered":"2024-04-26T18:22:59.011004+0000","last_clean":"2024-04-26T18:22:59.011004+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:22:59.011004+0000","last_undegraded":"2024-04-26T18:22:59.011004+0000","last_fullsized":"2024-04-26T18:22:59.011004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089770+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:22:59.089770+0000","last_peered":"2024-04-26T18:22:59.089770+0000","last_clean":"2024-04-26T18:22:59.089770+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:22:59.089770+0000","last_undegraded":"2024-04-26T18:22:59.089770+0000","last_fullsized":"2024-04-26T18:22:59.089770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294503+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:22:59.294503+0000","last_peered":"2024-04-26T18:22:59.294503+0000","last_clean":"2024-04-26T18:22:59.294503+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:22:59.294503+0000","last_undegraded":"2024-04-26T18:22:59.294503+0000","last_fullsized":"2024-04-26T18:22:59.294503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.434093+0000","last_change":"2024-04-26T18:21:58.050914+0000","last_active":"2024-04-26T18:22:59.434093+0000","last_peered":"2024-04-26T18:22:59.434093+0000","last_clean":"2024-04-26T18:22:59.434093+0000","last_became_active":"2024-04-26T18:21:58.050400+0000","last_became_peered":"2024-04-26T18:21:58.050400+0000","last_unstale":"2024-04-26T18:22:59.434093+0000","last_undegraded":"2024-04-26T18:22:59.434093+0000","last_fullsized":"2024-04-26T18:22:59.434093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:37.789853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011330+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:22:59.011330+0000","last_peered":"2024-04-26T18:22:59.011330+0000","last_clean":"2024-04-26T18:22:59.011330+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:22:59.011330+0000","last_undegraded":"2024-04-26T18:22:59.011330+0000","last_fullsized":"2024-04-26T18:22:59.011330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090045+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:22:59.090045+0000","last_peered":"2024-04-26T18:22:59.090045+0000","last_clean":"2024-04-26T18:22:59.090045+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:22:59.090045+0000","last_undegraded":"2024-04-26T18:22:59.090045+0000","last_fullsized":"2024-04-26T18:22:59.090045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014444+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:22:59.014444+0000","last_peered":"2024-04-26T18:22:59.014444+0000","last_clean":"2024-04-26T18:22:59.014444+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:22:59.014444+0000","last_undegraded":"2024-04-26T18:22:59.014444+0000","last_fullsized":"2024-04-26T18:22:59.014444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295277+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:22:59.295277+0000","last_peered":"2024-04-26T18:22:59.295277+0000","last_clean":"2024-04-26T18:22:59.295277+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:22:59.295277+0000","last_undegraded":"2024-04-26T18:22:59.295277+0000","last_fullsized":"2024-04-26T18:22:59.295277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.013958+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:22:59.013958+0000","last_peered":"2024-04-26T18:22:59.013958+0000","last_clean":"2024-04-26T18:22:59.013958+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:22:59.013958+0000","last_undegraded":"2024-04-26T18:22:59.013958+0000","last_fullsized":"2024-04-26T18:22:59.013958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014003+0000","last_change":"2024-04-26T18:21:58.051267+0000","last_active":"2024-04-26T18:22:59.014003+0000","last_peered":"2024-04-26T18:22:59.014003+0000","last_clean":"2024-04-26T18:22:59.014003+0000","last_became_active":"2024-04-26T18:21:58.050912+0000","last_became_peered":"2024-04-26T18:21:58.050912+0000","last_unstale":"2024-04-26T18:22:59.014003+0000","last_undegraded":"2024-04-26T18:22:59.014003+0000","last_fullsized":"2024-04-26T18:22:59.014003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:42.244892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433577+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:22:59.433577+0000","last_peered":"2024-04-26T18:22:59.433577+0000","last_clean":"2024-04-26T18:22:59.433577+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:22:59.433577+0000","last_undegraded":"2024-04-26T18:22:59.433577+0000","last_fullsized":"2024-04-26T18:22:59.433577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012460+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:22:59.012460+0000","last_peered":"2024-04-26T18:22:59.012460+0000","last_clean":"2024-04-26T18:22:59.012460+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:22:59.012460+0000","last_undegraded":"2024-04-26T18:22:59.012460+0000","last_fullsized":"2024-04-26T18:22:59.012460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009906+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:22:59.009906+0000","last_peered":"2024-04-26T18:22:59.009906+0000","last_clean":"2024-04-26T18:22:59.009906+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:22:59.009906+0000","last_undegraded":"2024-04-26T18:22:59.009906+0000","last_fullsized":"2024-04-26T18:22:59.009906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295351+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:22:59.295351+0000","last_peered":"2024-04-26T18:22:59.295351+0000","last_clean":"2024-04-26T18:22:59.295351+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:22:59.295351+0000","last_undegraded":"2024-04-26T18:22:59.295351+0000","last_fullsized":"2024-04-26T18:22:59.295351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009905+0000","last_change":"2024-04-26T18:21:58.059181+0000","last_active":"2024-04-26T18:22:59.009905+0000","last_peered":"2024-04-26T18:22:59.009905+0000","last_clean":"2024-04-26T18:22:59.009905+0000","last_became_active":"2024-04-26T18:21:58.051741+0000","last_became_peered":"2024-04-26T18:21:58.051741+0000","last_unstale":"2024-04-26T18:22:59.009905+0000","last_undegraded":"2024-04-26T18:22:59.009905+0000","last_fullsized":"2024-04-26T18:22:59.009905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:55.251611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294824+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:22:59.294824+0000","last_peered":"2024-04-26T18:22:59.294824+0000","last_clean":"2024-04-26T18:22:59.294824+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:22:59.294824+0000","last_undegraded":"2024-04-26T18:22:59.294824+0000","last_fullsized":"2024-04-26T18:22:59.294824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295159+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:22:59.295159+0000","last_peered":"2024-04-26T18:22:59.295159+0000","last_clean":"2024-04-26T18:22:59.295159+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:22:59.295159+0000","last_undegraded":"2024-04-26T18:22:59.295159+0000","last_fullsized":"2024-04-26T18:22:59.295159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014407+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:22:59.014407+0000","last_peered":"2024-04-26T18:22:59.014407+0000","last_clean":"2024-04-26T18:22:59.014407+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:22:59.014407+0000","last_undegraded":"2024-04-26T18:22:59.014407+0000","last_fullsized":"2024-04-26T18:22:59.014407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010702+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:22:59.010702+0000","last_peered":"2024-04-26T18:22:59.010702+0000","last_clean":"2024-04-26T18:22:59.010702+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:22:59.010702+0000","last_undegraded":"2024-04-26T18:22:59.010702+0000","last_fullsized":"2024-04-26T18:22:59.010702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433715+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:22:59.433715+0000","last_peered":"2024-04-26T18:22:59.433715+0000","last_clean":"2024-04-26T18:22:59.433715+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:22:59.433715+0000","last_undegraded":"2024-04-26T18:22:59.433715+0000","last_fullsized":"2024-04-26T18:22:59.433715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010791+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:22:59.010791+0000","last_peered":"2024-04-26T18:22:59.010791+0000","last_clean":"2024-04-26T18:22:59.010791+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:22:59.010791+0000","last_undegraded":"2024-04-26T18:22:59.010791+0000","last_fullsized":"2024-04-26T18:22:59.010791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095479+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:22:59.095479+0000","last_peered":"2024-04-26T18:22:59.095479+0000","last_clean":"2024-04-26T18:22:59.095479+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:22:59.095479+0000","last_undegraded":"2024-04-26T18:22:59.095479+0000","last_fullsized":"2024-04-26T18:22:59.095479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:03.101624+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:23:03.101624+0000","last_peered":"2024-04-26T18:23:03.101624+0000","last_clean":"2024-04-26T18:23:03.101624+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:23:03.101624+0000","last_undegraded":"2024-04-26T18:23:03.101624+0000","last_fullsized":"2024-04-26T18:23:03.101624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010373+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:22:59.010373+0000","last_peered":"2024-04-26T18:22:59.010373+0000","last_clean":"2024-04-26T18:22:59.010373+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:22:59.010373+0000","last_undegraded":"2024-04-26T18:22:59.010373+0000","last_fullsized":"2024-04-26T18:22:59.010373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010863+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:22:59.010863+0000","last_peered":"2024-04-26T18:22:59.010863+0000","last_clean":"2024-04-26T18:22:59.010863+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:22:59.010863+0000","last_undegraded":"2024-04-26T18:22:59.010863+0000","last_fullsized":"2024-04-26T18:22:59.010863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095244+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:22:59.095244+0000","last_peered":"2024-04-26T18:22:59.095244+0000","last_clean":"2024-04-26T18:22:59.095244+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:22:59.095244+0000","last_undegraded":"2024-04-26T18:22:59.095244+0000","last_fullsized":"2024-04-26T18:22:59.095244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295232+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:22:59.295232+0000","last_peered":"2024-04-26T18:22:59.295232+0000","last_clean":"2024-04-26T18:22:59.295232+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:22:59.295232+0000","last_undegraded":"2024-04-26T18:22:59.295232+0000","last_fullsized":"2024-04-26T18:22:59.295232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294608+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:22:59.294608+0000","last_peered":"2024-04-26T18:22:59.294608+0000","last_clean":"2024-04-26T18:22:59.294608+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:22:59.294608+0000","last_undegraded":"2024-04-26T18:22:59.294608+0000","last_fullsized":"2024-04-26T18:22:59.294608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095138+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:22:59.095138+0000","last_peered":"2024-04-26T18:22:59.095138+0000","last_clean":"2024-04-26T18:22:59.095138+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:22:59.095138+0000","last_undegraded":"2024-04-26T18:22:59.095138+0000","last_fullsized":"2024-04-26T18:22:59.095138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009667+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:22:59.009667+0000","last_peered":"2024-04-26T18:22:59.009667+0000","last_clean":"2024-04-26T18:22:59.009667+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:22:59.009667+0000","last_undegraded":"2024-04-26T18:22:59.009667+0000","last_fullsized":"2024-04-26T18:22:59.009667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:04.102474+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:23:04.102474+0000","last_peered":"2024-04-26T18:23:04.102474+0000","last_clean":"2024-04-26T18:23:04.102474+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:23:04.102474+0000","last_undegraded":"2024-04-26T18:23:04.102474+0000","last_fullsized":"2024-04-26T18:23:04.102474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433989+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:22:59.433989+0000","last_peered":"2024-04-26T18:22:59.433989+0000","last_clean":"2024-04-26T18:22:59.433989+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:22:59.433989+0000","last_undegraded":"2024-04-26T18:22:59.433989+0000","last_fullsized":"2024-04-26T18:22:59.433989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090073+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:22:59.090073+0000","last_peered":"2024-04-26T18:22:59.090073+0000","last_clean":"2024-04-26T18:22:59.090073+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:22:59.090073+0000","last_undegraded":"2024-04-26T18:22:59.090073+0000","last_fullsized":"2024-04-26T18:22:59.090073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014401+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:22:59.014401+0000","last_peered":"2024-04-26T18:22:59.014401+0000","last_clean":"2024-04-26T18:22:59.014401+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:22:59.014401+0000","last_undegraded":"2024-04-26T18:22:59.014401+0000","last_fullsized":"2024-04-26T18:22:59.014401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095634+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:22:59.095634+0000","last_peered":"2024-04-26T18:22:59.095634+0000","last_clean":"2024-04-26T18:22:59.095634+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:22:59.095634+0000","last_undegraded":"2024-04-26T18:22:59.095634+0000","last_fullsized":"2024-04-26T18:22:59.095634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433724+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:22:59.433724+0000","last_peered":"2024-04-26T18:22:59.433724+0000","last_clean":"2024-04-26T18:22:59.433724+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:22:59.433724+0000","last_undegraded":"2024-04-26T18:22:59.433724+0000","last_fullsized":"2024-04-26T18:22:59.433724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":269,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:05.104003+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:23:05.104003+0000","last_peered":"2024-04-26T18:23:05.104003+0000","last_clean":"2024-04-26T18:23:05.104003+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:23:05.104003+0000","last_undegraded":"2024-04-26T18:23:05.104003+0000","last_fullsized":"2024-04-26T18:23:05.104003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":163,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010529+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:22:59.010529+0000","last_peered":"2024-04-26T18:22:59.010529+0000","last_clean":"2024-04-26T18:22:59.010529+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:22:59.010529+0000","last_undegraded":"2024-04-26T18:22:59.010529+0000","last_fullsized":"2024-04-26T18:22:59.010529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295100+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:22:59.295100+0000","last_peered":"2024-04-26T18:22:59.295100+0000","last_clean":"2024-04-26T18:22:59.295100+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:22:59.295100+0000","last_undegraded":"2024-04-26T18:22:59.295100+0000","last_fullsized":"2024-04-26T18:22:59.295100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295423+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:22:59.295423+0000","last_peered":"2024-04-26T18:22:59.295423+0000","last_clean":"2024-04-26T18:22:59.295423+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:22:59.295423+0000","last_undegraded":"2024-04-26T18:22:59.295423+0000","last_fullsized":"2024-04-26T18:22:59.295423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095205+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:22:59.095205+0000","last_peered":"2024-04-26T18:22:59.095205+0000","last_clean":"2024-04-26T18:22:59.095205+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:22:59.095205+0000","last_undegraded":"2024-04-26T18:22:59.095205+0000","last_fullsized":"2024-04-26T18:22:59.095205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089945+0000","last_change":"2024-04-26T18:21:58.049111+0000","last_active":"2024-04-26T18:22:59.089945+0000","last_peered":"2024-04-26T18:22:59.089945+0000","last_clean":"2024-04-26T18:22:59.089945+0000","last_became_active":"2024-04-26T18:21:58.048723+0000","last_became_peered":"2024-04-26T18:21:58.048723+0000","last_unstale":"2024-04-26T18:22:59.089945+0000","last_undegraded":"2024-04-26T18:22:59.089945+0000","last_fullsized":"2024-04-26T18:22:59.089945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:26.107528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:03.328115+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:23:03.328115+0000","last_peered":"2024-04-26T18:23:03.328115+0000","last_clean":"2024-04-26T18:23:03.328115+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:23:03.328115+0000","last_undegraded":"2024-04-26T18:23:03.328115+0000","last_fullsized":"2024-04-26T18:23:03.328115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089605+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:22:59.089605+0000","last_peered":"2024-04-26T18:22:59.089605+0000","last_clean":"2024-04-26T18:22:59.089605+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:22:59.089605+0000","last_undegraded":"2024-04-26T18:22:59.089605+0000","last_fullsized":"2024-04-26T18:22:59.089605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.027403+0000","last_change":"2024-04-26T18:22:59.027035+0000","last_active":"2024-04-26T18:22:59.027403+0000","last_peered":"2024-04-26T18:22:59.027403+0000","last_clean":"2024-04-26T18:22:59.027403+0000","last_became_active":"2024-04-26T18:22:59.015160+0000","last_became_peered":"2024-04-26T18:22:59.015160+0000","last_unstale":"2024-04-26T18:22:59.027403+0000","last_undegraded":"2024-04-26T18:22:59.027403+0000","last_fullsized":"2024-04-26T18:22:59.027403+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:04.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009669+0000","last_change":"2024-04-26T18:21:58.052169+0000","last_active":"2024-04-26T18:22:59.009669+0000","last_peered":"2024-04-26T18:22:59.009669+0000","last_clean":"2024-04-26T18:22:59.009669+0000","last_became_active":"2024-04-26T18:21:58.051756+0000","last_became_peered":"2024-04-26T18:21:58.051756+0000","last_unstale":"2024-04-26T18:22:59.009669+0000","last_undegraded":"2024-04-26T18:22:59.009669+0000","last_fullsized":"2024-04-26T18:22:59.009669+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:11.730323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010735+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:22:59.010735+0000","last_peered":"2024-04-26T18:22:59.010735+0000","last_clean":"2024-04-26T18:22:59.010735+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:22:59.010735+0000","last_undegraded":"2024-04-26T18:22:59.010735+0000","last_fullsized":"2024-04-26T18:22:59.010735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012271+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:22:59.012271+0000","last_peered":"2024-04-26T18:22:59.012271+0000","last_clean":"2024-04-26T18:22:59.012271+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:22:59.012271+0000","last_undegraded":"2024-04-26T18:22:59.012271+0000","last_fullsized":"2024-04-26T18:22:59.012271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009818+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:22:59.009818+0000","last_peered":"2024-04-26T18:22:59.009818+0000","last_clean":"2024-04-26T18:22:59.009818+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:22:59.009818+0000","last_undegraded":"2024-04-26T18:22:59.009818+0000","last_fullsized":"2024-04-26T18:22:59.009818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011128+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:22:59.011128+0000","last_peered":"2024-04-26T18:22:59.011128+0000","last_clean":"2024-04-26T18:22:59.011128+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:22:59.011128+0000","last_undegraded":"2024-04-26T18:22:59.011128+0000","last_fullsized":"2024-04-26T18:22:59.011128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089834+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:22:59.089834+0000","last_peered":"2024-04-26T18:22:59.089834+0000","last_clean":"2024-04-26T18:22:59.089834+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:22:59.089834+0000","last_undegraded":"2024-04-26T18:22:59.089834+0000","last_fullsized":"2024-04-26T18:22:59.089834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010827+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:22:59.010827+0000","last_peered":"2024-04-26T18:22:59.010827+0000","last_clean":"2024-04-26T18:22:59.010827+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:22:59.010827+0000","last_undegraded":"2024-04-26T18:22:59.010827+0000","last_fullsized":"2024-04-26T18:22:59.010827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095605+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:22:59.095605+0000","last_peered":"2024-04-26T18:22:59.095605+0000","last_clean":"2024-04-26T18:22:59.095605+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:22:59.095605+0000","last_undegraded":"2024-04-26T18:22:59.095605+0000","last_fullsized":"2024-04-26T18:22:59.095605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:04.329059+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:23:04.329059+0000","last_peered":"2024-04-26T18:23:04.329059+0000","last_clean":"2024-04-26T18:23:04.329059+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:23:04.329059+0000","last_undegraded":"2024-04-26T18:23:04.329059+0000","last_fullsized":"2024-04-26T18:23:04.329059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294859+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:22:59.294859+0000","last_peered":"2024-04-26T18:22:59.294859+0000","last_clean":"2024-04-26T18:22:59.294859+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:22:59.294859+0000","last_undegraded":"2024-04-26T18:22:59.294859+0000","last_fullsized":"2024-04-26T18:22:59.294859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014293+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:22:59.014293+0000","last_peered":"2024-04-26T18:22:59.014293+0000","last_clean":"2024-04-26T18:22:59.014293+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:22:59.014293+0000","last_undegraded":"2024-04-26T18:22:59.014293+0000","last_fullsized":"2024-04-26T18:22:59.014293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089701+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:22:59.089701+0000","last_peered":"2024-04-26T18:22:59.089701+0000","last_clean":"2024-04-26T18:22:59.089701+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:22:59.089701+0000","last_undegraded":"2024-04-26T18:22:59.089701+0000","last_fullsized":"2024-04-26T18:22:59.089701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294996+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:22:59.294996+0000","last_peered":"2024-04-26T18:22:59.294996+0000","last_clean":"2024-04-26T18:22:59.294996+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:22:59.294996+0000","last_undegraded":"2024-04-26T18:22:59.294996+0000","last_fullsized":"2024-04-26T18:22:59.294996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089835+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:22:59.089835+0000","last_peered":"2024-04-26T18:22:59.089835+0000","last_clean":"2024-04-26T18:22:59.089835+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:22:59.089835+0000","last_undegraded":"2024-04-26T18:22:59.089835+0000","last_fullsized":"2024-04-26T18:22:59.089835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010551+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:22:59.010551+0000","last_peered":"2024-04-26T18:22:59.010551+0000","last_clean":"2024-04-26T18:22:59.010551+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:22:59.010551+0000","last_undegraded":"2024-04-26T18:22:59.010551+0000","last_fullsized":"2024-04-26T18:22:59.010551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295541+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:22:59.295541+0000","last_peered":"2024-04-26T18:22:59.295541+0000","last_clean":"2024-04-26T18:22:59.295541+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:22:59.295541+0000","last_undegraded":"2024-04-26T18:22:59.295541+0000","last_fullsized":"2024-04-26T18:22:59.295541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009966+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:22:59.009966+0000","last_peered":"2024-04-26T18:22:59.009966+0000","last_clean":"2024-04-26T18:22:59.009966+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:22:59.009966+0000","last_undegraded":"2024-04-26T18:22:59.009966+0000","last_fullsized":"2024-04-26T18:22:59.009966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295245+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:22:59.295245+0000","last_peered":"2024-04-26T18:22:59.295245+0000","last_clean":"2024-04-26T18:22:59.295245+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:22:59.295245+0000","last_undegraded":"2024-04-26T18:22:59.295245+0000","last_fullsized":"2024-04-26T18:22:59.295245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014653+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:22:59.014653+0000","last_peered":"2024-04-26T18:22:59.014653+0000","last_clean":"2024-04-26T18:22:59.014653+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:22:59.014653+0000","last_undegraded":"2024-04-26T18:22:59.014653+0000","last_fullsized":"2024-04-26T18:22:59.014653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014363+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:22:59.014363+0000","last_peered":"2024-04-26T18:22:59.014363+0000","last_clean":"2024-04-26T18:22:59.014363+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:22:59.014363+0000","last_undegraded":"2024-04-26T18:22:59.014363+0000","last_fullsized":"2024-04-26T18:22:59.014363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433592+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:22:59.433592+0000","last_peered":"2024-04-26T18:22:59.433592+0000","last_clean":"2024-04-26T18:22:59.433592+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:22:59.433592+0000","last_undegraded":"2024-04-26T18:22:59.433592+0000","last_fullsized":"2024-04-26T18:22:59.433592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010291+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:22:59.010291+0000","last_peered":"2024-04-26T18:22:59.010291+0000","last_clean":"2024-04-26T18:22:59.010291+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:22:59.010291+0000","last_undegraded":"2024-04-26T18:22:59.010291+0000","last_fullsized":"2024-04-26T18:22:59.010291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294569+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:22:59.294569+0000","last_peered":"2024-04-26T18:22:59.294569+0000","last_clean":"2024-04-26T18:22:59.294569+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:22:59.294569+0000","last_undegraded":"2024-04-26T18:22:59.294569+0000","last_fullsized":"2024-04-26T18:22:59.294569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012832+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:22:59.012832+0000","last_peered":"2024-04-26T18:22:59.012832+0000","last_clean":"2024-04-26T18:22:59.012832+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:22:59.012832+0000","last_undegraded":"2024-04-26T18:22:59.012832+0000","last_fullsized":"2024-04-26T18:22:59.012832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012067+0000","last_change":"2024-04-26T18:21:58.057410+0000","last_active":"2024-04-26T18:22:59.012067+0000","last_peered":"2024-04-26T18:22:59.012067+0000","last_clean":"2024-04-26T18:22:59.012067+0000","last_became_active":"2024-04-26T18:21:58.054453+0000","last_became_peered":"2024-04-26T18:21:58.054453+0000","last_unstale":"2024-04-26T18:22:59.012067+0000","last_undegraded":"2024-04-26T18:22:59.012067+0000","last_fullsized":"2024-04-26T18:22:59.012067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:22.856834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294758+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:22:59.294758+0000","last_peered":"2024-04-26T18:22:59.294758+0000","last_clean":"2024-04-26T18:22:59.294758+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:22:59.294758+0000","last_undegraded":"2024-04-26T18:22:59.294758+0000","last_fullsized":"2024-04-26T18:22:59.294758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010476+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:22:59.010476+0000","last_peered":"2024-04-26T18:22:59.010476+0000","last_clean":"2024-04-26T18:22:59.010476+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:22:59.010476+0000","last_undegraded":"2024-04-26T18:22:59.010476+0000","last_fullsized":"2024-04-26T18:22:59.010476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089868+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:22:59.089868+0000","last_peered":"2024-04-26T18:22:59.089868+0000","last_clean":"2024-04-26T18:22:59.089868+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:22:59.089868+0000","last_undegraded":"2024-04-26T18:22:59.089868+0000","last_fullsized":"2024-04-26T18:22:59.089868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014487+0000","last_change":"2024-04-26T18:21:58.069561+0000","last_active":"2024-04-26T18:22:59.014487+0000","last_peered":"2024-04-26T18:22:59.014487+0000","last_clean":"2024-04-26T18:22:59.014487+0000","last_became_active":"2024-04-26T18:21:58.053199+0000","last_became_peered":"2024-04-26T18:21:58.053199+0000","last_unstale":"2024-04-26T18:22:59.014487+0000","last_undegraded":"2024-04-26T18:22:59.014487+0000","last_fullsized":"2024-04-26T18:22:59.014487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:27.166045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090184+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:22:59.090184+0000","last_peered":"2024-04-26T18:22:59.090184+0000","last_clean":"2024-04-26T18:22:59.090184+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:22:59.090184+0000","last_undegraded":"2024-04-26T18:22:59.090184+0000","last_fullsized":"2024-04-26T18:22:59.090184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095420+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:22:59.095420+0000","last_peered":"2024-04-26T18:22:59.095420+0000","last_clean":"2024-04-26T18:22:59.095420+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:22:59.095420+0000","last_undegraded":"2024-04-26T18:22:59.095420+0000","last_fullsized":"2024-04-26T18:22:59.095420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089990+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:22:59.089990+0000","last_peered":"2024-04-26T18:22:59.089990+0000","last_clean":"2024-04-26T18:22:59.089990+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:22:59.089990+0000","last_undegraded":"2024-04-26T18:22:59.089990+0000","last_fullsized":"2024-04-26T18:22:59.089990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295216+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:22:59.295216+0000","last_peered":"2024-04-26T18:22:59.295216+0000","last_clean":"2024-04-26T18:22:59.295216+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:22:59.295216+0000","last_undegraded":"2024-04-26T18:22:59.295216+0000","last_fullsized":"2024-04-26T18:22:59.295216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090164+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:22:59.090164+0000","last_peered":"2024-04-26T18:22:59.090164+0000","last_clean":"2024-04-26T18:22:59.090164+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:22:59.090164+0000","last_undegraded":"2024-04-26T18:22:59.090164+0000","last_fullsized":"2024-04-26T18:22:59.090164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012833+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:22:59.012833+0000","last_peered":"2024-04-26T18:22:59.012833+0000","last_clean":"2024-04-26T18:22:59.012833+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:22:59.012833+0000","last_undegraded":"2024-04-26T18:22:59.012833+0000","last_fullsized":"2024-04-26T18:22:59.012833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295045+0000","last_change":"2024-04-26T18:21:58.052815+0000","last_active":"2024-04-26T18:22:59.295045+0000","last_peered":"2024-04-26T18:22:59.295045+0000","last_clean":"2024-04-26T18:22:59.295045+0000","last_became_active":"2024-04-26T18:21:58.052405+0000","last_became_peered":"2024-04-26T18:21:58.052405+0000","last_unstale":"2024-04-26T18:22:59.295045+0000","last_undegraded":"2024-04-26T18:22:59.295045+0000","last_fullsized":"2024-04-26T18:22:59.295045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:29.289089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294975+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:22:59.294975+0000","last_peered":"2024-04-26T18:22:59.294975+0000","last_clean":"2024-04-26T18:22:59.294975+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:22:59.294975+0000","last_undegraded":"2024-04-26T18:22:59.294975+0000","last_fullsized":"2024-04-26T18:22:59.294975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.013993+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:22:59.013993+0000","last_peered":"2024-04-26T18:22:59.013993+0000","last_clean":"2024-04-26T18:22:59.013993+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:22:59.013993+0000","last_undegraded":"2024-04-26T18:22:59.013993+0000","last_fullsized":"2024-04-26T18:22:59.013993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295554+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:22:59.295554+0000","last_peered":"2024-04-26T18:22:59.295554+0000","last_clean":"2024-04-26T18:22:59.295554+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:22:59.295554+0000","last_undegraded":"2024-04-26T18:22:59.295554+0000","last_fullsized":"2024-04-26T18:22:59.295554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012283+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:22:59.012283+0000","last_peered":"2024-04-26T18:22:59.012283+0000","last_clean":"2024-04-26T18:22:59.012283+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:22:59.012283+0000","last_undegraded":"2024-04-26T18:22:59.012283+0000","last_fullsized":"2024-04-26T18:22:59.012283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295422+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:22:59.295422+0000","last_peered":"2024-04-26T18:22:59.295422+0000","last_clean":"2024-04-26T18:22:59.295422+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:22:59.295422+0000","last_undegraded":"2024-04-26T18:22:59.295422+0000","last_fullsized":"2024-04-26T18:22:59.295422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295137+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:22:59.295137+0000","last_peered":"2024-04-26T18:22:59.295137+0000","last_clean":"2024-04-26T18:22:59.295137+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:22:59.295137+0000","last_undegraded":"2024-04-26T18:22:59.295137+0000","last_fullsized":"2024-04-26T18:22:59.295137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095207+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:22:59.095207+0000","last_peered":"2024-04-26T18:22:59.095207+0000","last_clean":"2024-04-26T18:22:59.095207+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:22:59.095207+0000","last_undegraded":"2024-04-26T18:22:59.095207+0000","last_fullsized":"2024-04-26T18:22:59.095207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009525+0000","last_change":"2024-04-26T18:21:58.051597+0000","last_active":"2024-04-26T18:22:59.009525+0000","last_peered":"2024-04-26T18:22:59.009525+0000","last_clean":"2024-04-26T18:22:59.009525+0000","last_became_active":"2024-04-26T18:21:58.051139+0000","last_became_peered":"2024-04-26T18:21:58.051139+0000","last_unstale":"2024-04-26T18:22:59.009525+0000","last_undegraded":"2024-04-26T18:22:59.009525+0000","last_fullsized":"2024-04-26T18:22:59.009525+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.289734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433776+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:22:59.433776+0000","last_peered":"2024-04-26T18:22:59.433776+0000","last_clean":"2024-04-26T18:22:59.433776+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:22:59.433776+0000","last_undegraded":"2024-04-26T18:22:59.433776+0000","last_fullsized":"2024-04-26T18:22:59.433776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295406+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:22:59.295406+0000","last_peered":"2024-04-26T18:22:59.295406+0000","last_clean":"2024-04-26T18:22:59.295406+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:22:59.295406+0000","last_undegraded":"2024-04-26T18:22:59.295406+0000","last_fullsized":"2024-04-26T18:22:59.295406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012125+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:22:59.012125+0000","last_peered":"2024-04-26T18:22:59.012125+0000","last_clean":"2024-04-26T18:22:59.012125+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:22:59.012125+0000","last_undegraded":"2024-04-26T18:22:59.012125+0000","last_fullsized":"2024-04-26T18:22:59.012125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":620,"num_read_kb":379,"num_write":171,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1276,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40508,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704644,"statfs":{"total":95995035648,"available":95953555456,"internally_reserved":0,"allocated":1028096,"data_stored":497190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1149155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":497314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":496540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1089305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":496703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":557674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":497974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40576,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704576,"statfs":{"total":95995035648,"available":95953485824,"internally_reserved":0,"allocated":1097728,"data_stored":558280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":497014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1088989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":44,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:23:08.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:23:08.312 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:08.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:08 smithi170 ceph-mon[32697]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 588 B/s wr, 7 op/s 2024-04-26T18:23:08.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:08 smithi170 ceph-mon[32697]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:08.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:08 smithi037 ceph-mon[29171]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 588 B/s wr, 7 op/s 2024-04-26T18:23:08.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:08 smithi037 ceph-mon[29171]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:08.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:08 smithi120 ceph-mon[32244]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 588 B/s wr, 7 op/s 2024-04-26T18:23:08.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:08 smithi120 ceph-mon[32244]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 -- 172.21.15.37:0/310694941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99041094d0 msgr2=0x7f99041098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 --2- 172.21.15.37:0/310694941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99041094d0 0x7f99041098b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9900009990 tx=0x7f9900009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 -- 172.21.15.37:0/310694941 shutdown_connections 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 --2- 172.21.15.37:0/310694941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f990406c590 0x7f9904110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 --2- 172.21.15.37:0/310694941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99041094d0 0x7f99041098b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 --2- 172.21.15.37:0/310694941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406bbd0 0x7f990406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 -- 172.21.15.37:0/310694941 >> 172.21.15.37:0/310694941 conn(0x7f99040793c0 msgr2=0x7f99040797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 -- 172.21.15.37:0/310694941 shutdown_connections 2024-04-26T18:23:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.926+0000 7f990c01f700 1 -- 172.21.15.37:0/310694941 wait complete. 2024-04-26T18:23:09.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 Processor -- start 2024-04-26T18:23:09.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 -- start start 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406c590 0x7f990419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99041094d0 0x7f99041999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9904113660 con 0x7f990406bbd0 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99041134e0 con 0x7f99041094d0 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990c01f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99041137e0 con 0x7f990406c590 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f9909dbb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f9909dbb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49940/0 (socket says 172.21.15.37:49940) 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f9909dbb700 1 -- 172.21.15.37:0/377533330 learned_addr learned my addr 172.21.15.37:0/377533330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.927+0000 7f990a5bc700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99041094d0 0x7f99041999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f99095ba700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406c590 0x7f990419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 -- 172.21.15.37:0/377533330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406c590 msgr2=0x7f990419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406c590 0x7f990419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 -- 172.21.15.37:0/377533330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99041094d0 msgr2=0x7f99041999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99041094d0 0x7f99041999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 -- 172.21.15.37:0/377533330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990419a120 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f9909dbb700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f99000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900027070 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9900003f20 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990419a3b0 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900022cf0 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.928+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a6660 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.930+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990410ba40 con 0x7f990406bbd0 2024-04-26T18:23:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.930+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f990002a070 con 0x7f990406bbd0 2024-04-26T18:23:09.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.931+0000 7f98faffd700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 0x7f98f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:09.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.931+0000 7f99095ba700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 0x7f98f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:09.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.931+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f99000d1230 con 0x7f990406bbd0 2024-04-26T18:23:09.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.931+0000 7f99095ba700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 0x7f98f00a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006fd0 tx=0x7f98f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:09.936+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9900013070 con 0x7f990406bbd0 2024-04-26T18:23:10.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.143+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f990419ae80 con 0x7f98f00a3c40 2024-04-26T18:23:10.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.150+0000 7f98faffd700 1 -- 172.21.15.37:0/377533330 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258853 (secure 0 0 0) 0x7f990419ae80 con 0x7f98f00a3c40 2024-04-26T18:23:10.153 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 msgr2=0x7f98f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 0x7f98f00a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006fd0 tx=0x7f98f4008040 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 msgr2=0x7f990419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c00 tx=0x7f99000047b0 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 shutdown_connections 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f98f00a3c40 0x7f98f00a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99041094d0 0x7f99041999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406bbd0 0x7f990419f790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 --2- 172.21.15.37:0/377533330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f990406c590 0x7f990419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 >> 172.21.15.37:0/377533330 conn(0x7f99040793c0 msgr2=0x7f9904100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 shutdown_connections 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:10.153+0000 7f990c01f700 1 -- 172.21.15.37:0/377533330 wait complete. 2024-04-26T18:23:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:23:10.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:10 smithi037 ceph-mon[29171]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T18:23:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:10 smithi170 ceph-mon[32697]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T18:23:10.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:10 smithi120 ceph-mon[32244]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T18:23:10.978 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-04-26T18:23:08.702775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":756,"num_read_kb":498,"num_write":354,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182012,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384116,"kb_used_data":14076,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557708,"statfs":{"total":1151940427776,"available":1151547092992,"internally_reserved":0,"allocated":14413824,"data_stored":7923972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002572"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012366+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:22:59.012366+0000","last_peered":"2024-04-26T18:22:59.012366+0000","last_clean":"2024-04-26T18:22:59.012366+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:22:59.012366+0000","last_undegraded":"2024-04-26T18:22:59.012366+0000","last_fullsized":"2024-04-26T18:22:59.012366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014261+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:22:59.014261+0000","last_peered":"2024-04-26T18:22:59.014261+0000","last_clean":"2024-04-26T18:22:59.014261+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:22:59.014261+0000","last_undegraded":"2024-04-26T18:22:59.014261+0000","last_fullsized":"2024-04-26T18:22:59.014261+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014249+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:22:59.014249+0000","last_peered":"2024-04-26T18:22:59.014249+0000","last_clean":"2024-04-26T18:22:59.014249+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:22:59.014249+0000","last_undegraded":"2024-04-26T18:22:59.014249+0000","last_fullsized":"2024-04-26T18:22:59.014249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014594+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:22:59.014594+0000","last_peered":"2024-04-26T18:22:59.014594+0000","last_clean":"2024-04-26T18:22:59.014594+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:22:59.014594+0000","last_undegraded":"2024-04-26T18:22:59.014594+0000","last_fullsized":"2024-04-26T18:22:59.014594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010368+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:22:59.010368+0000","last_peered":"2024-04-26T18:22:59.010368+0000","last_clean":"2024-04-26T18:22:59.010368+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:22:59.010368+0000","last_undegraded":"2024-04-26T18:22:59.010368+0000","last_fullsized":"2024-04-26T18:22:59.010368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011004+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:22:59.011004+0000","last_peered":"2024-04-26T18:22:59.011004+0000","last_clean":"2024-04-26T18:22:59.011004+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:22:59.011004+0000","last_undegraded":"2024-04-26T18:22:59.011004+0000","last_fullsized":"2024-04-26T18:22:59.011004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089770+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:22:59.089770+0000","last_peered":"2024-04-26T18:22:59.089770+0000","last_clean":"2024-04-26T18:22:59.089770+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:22:59.089770+0000","last_undegraded":"2024-04-26T18:22:59.089770+0000","last_fullsized":"2024-04-26T18:22:59.089770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294503+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:22:59.294503+0000","last_peered":"2024-04-26T18:22:59.294503+0000","last_clean":"2024-04-26T18:22:59.294503+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:22:59.294503+0000","last_undegraded":"2024-04-26T18:22:59.294503+0000","last_fullsized":"2024-04-26T18:22:59.294503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.434093+0000","last_change":"2024-04-26T18:21:58.050914+0000","last_active":"2024-04-26T18:22:59.434093+0000","last_peered":"2024-04-26T18:22:59.434093+0000","last_clean":"2024-04-26T18:22:59.434093+0000","last_became_active":"2024-04-26T18:21:58.050400+0000","last_became_peered":"2024-04-26T18:21:58.050400+0000","last_unstale":"2024-04-26T18:22:59.434093+0000","last_undegraded":"2024-04-26T18:22:59.434093+0000","last_fullsized":"2024-04-26T18:22:59.434093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:37.789853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011330+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:22:59.011330+0000","last_peered":"2024-04-26T18:22:59.011330+0000","last_clean":"2024-04-26T18:22:59.011330+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:22:59.011330+0000","last_undegraded":"2024-04-26T18:22:59.011330+0000","last_fullsized":"2024-04-26T18:22:59.011330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090045+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:22:59.090045+0000","last_peered":"2024-04-26T18:22:59.090045+0000","last_clean":"2024-04-26T18:22:59.090045+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:22:59.090045+0000","last_undegraded":"2024-04-26T18:22:59.090045+0000","last_fullsized":"2024-04-26T18:22:59.090045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014444+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:22:59.014444+0000","last_peered":"2024-04-26T18:22:59.014444+0000","last_clean":"2024-04-26T18:22:59.014444+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:22:59.014444+0000","last_undegraded":"2024-04-26T18:22:59.014444+0000","last_fullsized":"2024-04-26T18:22:59.014444+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295277+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:22:59.295277+0000","last_peered":"2024-04-26T18:22:59.295277+0000","last_clean":"2024-04-26T18:22:59.295277+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:22:59.295277+0000","last_undegraded":"2024-04-26T18:22:59.295277+0000","last_fullsized":"2024-04-26T18:22:59.295277+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.013958+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:22:59.013958+0000","last_peered":"2024-04-26T18:22:59.013958+0000","last_clean":"2024-04-26T18:22:59.013958+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:22:59.013958+0000","last_undegraded":"2024-04-26T18:22:59.013958+0000","last_fullsized":"2024-04-26T18:22:59.013958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014003+0000","last_change":"2024-04-26T18:21:58.051267+0000","last_active":"2024-04-26T18:22:59.014003+0000","last_peered":"2024-04-26T18:22:59.014003+0000","last_clean":"2024-04-26T18:22:59.014003+0000","last_became_active":"2024-04-26T18:21:58.050912+0000","last_became_peered":"2024-04-26T18:21:58.050912+0000","last_unstale":"2024-04-26T18:22:59.014003+0000","last_undegraded":"2024-04-26T18:22:59.014003+0000","last_fullsized":"2024-04-26T18:22:59.014003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:42.244892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433577+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:22:59.433577+0000","last_peered":"2024-04-26T18:22:59.433577+0000","last_clean":"2024-04-26T18:22:59.433577+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:22:59.433577+0000","last_undegraded":"2024-04-26T18:22:59.433577+0000","last_fullsized":"2024-04-26T18:22:59.433577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012460+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:22:59.012460+0000","last_peered":"2024-04-26T18:22:59.012460+0000","last_clean":"2024-04-26T18:22:59.012460+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:22:59.012460+0000","last_undegraded":"2024-04-26T18:22:59.012460+0000","last_fullsized":"2024-04-26T18:22:59.012460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009906+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:22:59.009906+0000","last_peered":"2024-04-26T18:22:59.009906+0000","last_clean":"2024-04-26T18:22:59.009906+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:22:59.009906+0000","last_undegraded":"2024-04-26T18:22:59.009906+0000","last_fullsized":"2024-04-26T18:22:59.009906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295351+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:22:59.295351+0000","last_peered":"2024-04-26T18:22:59.295351+0000","last_clean":"2024-04-26T18:22:59.295351+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:22:59.295351+0000","last_undegraded":"2024-04-26T18:22:59.295351+0000","last_fullsized":"2024-04-26T18:22:59.295351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009905+0000","last_change":"2024-04-26T18:21:58.059181+0000","last_active":"2024-04-26T18:22:59.009905+0000","last_peered":"2024-04-26T18:22:59.009905+0000","last_clean":"2024-04-26T18:22:59.009905+0000","last_became_active":"2024-04-26T18:21:58.051741+0000","last_became_peered":"2024-04-26T18:21:58.051741+0000","last_unstale":"2024-04-26T18:22:59.009905+0000","last_undegraded":"2024-04-26T18:22:59.009905+0000","last_fullsized":"2024-04-26T18:22:59.009905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:55.251611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294824+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:22:59.294824+0000","last_peered":"2024-04-26T18:22:59.294824+0000","last_clean":"2024-04-26T18:22:59.294824+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:22:59.294824+0000","last_undegraded":"2024-04-26T18:22:59.294824+0000","last_fullsized":"2024-04-26T18:22:59.294824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295159+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:22:59.295159+0000","last_peered":"2024-04-26T18:22:59.295159+0000","last_clean":"2024-04-26T18:22:59.295159+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:22:59.295159+0000","last_undegraded":"2024-04-26T18:22:59.295159+0000","last_fullsized":"2024-04-26T18:22:59.295159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014407+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:22:59.014407+0000","last_peered":"2024-04-26T18:22:59.014407+0000","last_clean":"2024-04-26T18:22:59.014407+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:22:59.014407+0000","last_undegraded":"2024-04-26T18:22:59.014407+0000","last_fullsized":"2024-04-26T18:22:59.014407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010702+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:22:59.010702+0000","last_peered":"2024-04-26T18:22:59.010702+0000","last_clean":"2024-04-26T18:22:59.010702+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:22:59.010702+0000","last_undegraded":"2024-04-26T18:22:59.010702+0000","last_fullsized":"2024-04-26T18:22:59.010702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433715+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:22:59.433715+0000","last_peered":"2024-04-26T18:22:59.433715+0000","last_clean":"2024-04-26T18:22:59.433715+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:22:59.433715+0000","last_undegraded":"2024-04-26T18:22:59.433715+0000","last_fullsized":"2024-04-26T18:22:59.433715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010791+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:22:59.010791+0000","last_peered":"2024-04-26T18:22:59.010791+0000","last_clean":"2024-04-26T18:22:59.010791+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:22:59.010791+0000","last_undegraded":"2024-04-26T18:22:59.010791+0000","last_fullsized":"2024-04-26T18:22:59.010791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095479+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:22:59.095479+0000","last_peered":"2024-04-26T18:22:59.095479+0000","last_clean":"2024-04-26T18:22:59.095479+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:22:59.095479+0000","last_undegraded":"2024-04-26T18:22:59.095479+0000","last_fullsized":"2024-04-26T18:22:59.095479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:03.101624+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:23:03.101624+0000","last_peered":"2024-04-26T18:23:03.101624+0000","last_clean":"2024-04-26T18:23:03.101624+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:23:03.101624+0000","last_undegraded":"2024-04-26T18:23:03.101624+0000","last_fullsized":"2024-04-26T18:23:03.101624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010373+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:22:59.010373+0000","last_peered":"2024-04-26T18:22:59.010373+0000","last_clean":"2024-04-26T18:22:59.010373+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:22:59.010373+0000","last_undegraded":"2024-04-26T18:22:59.010373+0000","last_fullsized":"2024-04-26T18:22:59.010373+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010863+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:22:59.010863+0000","last_peered":"2024-04-26T18:22:59.010863+0000","last_clean":"2024-04-26T18:22:59.010863+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:22:59.010863+0000","last_undegraded":"2024-04-26T18:22:59.010863+0000","last_fullsized":"2024-04-26T18:22:59.010863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095244+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:22:59.095244+0000","last_peered":"2024-04-26T18:22:59.095244+0000","last_clean":"2024-04-26T18:22:59.095244+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:22:59.095244+0000","last_undegraded":"2024-04-26T18:22:59.095244+0000","last_fullsized":"2024-04-26T18:22:59.095244+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295232+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:22:59.295232+0000","last_peered":"2024-04-26T18:22:59.295232+0000","last_clean":"2024-04-26T18:22:59.295232+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:22:59.295232+0000","last_undegraded":"2024-04-26T18:22:59.295232+0000","last_fullsized":"2024-04-26T18:22:59.295232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294608+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:22:59.294608+0000","last_peered":"2024-04-26T18:22:59.294608+0000","last_clean":"2024-04-26T18:22:59.294608+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:22:59.294608+0000","last_undegraded":"2024-04-26T18:22:59.294608+0000","last_fullsized":"2024-04-26T18:22:59.294608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095138+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:22:59.095138+0000","last_peered":"2024-04-26T18:22:59.095138+0000","last_clean":"2024-04-26T18:22:59.095138+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:22:59.095138+0000","last_undegraded":"2024-04-26T18:22:59.095138+0000","last_fullsized":"2024-04-26T18:22:59.095138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009667+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:22:59.009667+0000","last_peered":"2024-04-26T18:22:59.009667+0000","last_clean":"2024-04-26T18:22:59.009667+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:22:59.009667+0000","last_undegraded":"2024-04-26T18:22:59.009667+0000","last_fullsized":"2024-04-26T18:22:59.009667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:04.102474+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:23:04.102474+0000","last_peered":"2024-04-26T18:23:04.102474+0000","last_clean":"2024-04-26T18:23:04.102474+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:23:04.102474+0000","last_undegraded":"2024-04-26T18:23:04.102474+0000","last_fullsized":"2024-04-26T18:23:04.102474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433989+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:22:59.433989+0000","last_peered":"2024-04-26T18:22:59.433989+0000","last_clean":"2024-04-26T18:22:59.433989+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:22:59.433989+0000","last_undegraded":"2024-04-26T18:22:59.433989+0000","last_fullsized":"2024-04-26T18:22:59.433989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090073+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:22:59.090073+0000","last_peered":"2024-04-26T18:22:59.090073+0000","last_clean":"2024-04-26T18:22:59.090073+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:22:59.090073+0000","last_undegraded":"2024-04-26T18:22:59.090073+0000","last_fullsized":"2024-04-26T18:22:59.090073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014401+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:22:59.014401+0000","last_peered":"2024-04-26T18:22:59.014401+0000","last_clean":"2024-04-26T18:22:59.014401+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:22:59.014401+0000","last_undegraded":"2024-04-26T18:22:59.014401+0000","last_fullsized":"2024-04-26T18:22:59.014401+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095634+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:22:59.095634+0000","last_peered":"2024-04-26T18:22:59.095634+0000","last_clean":"2024-04-26T18:22:59.095634+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:22:59.095634+0000","last_undegraded":"2024-04-26T18:22:59.095634+0000","last_fullsized":"2024-04-26T18:22:59.095634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433724+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:22:59.433724+0000","last_peered":"2024-04-26T18:22:59.433724+0000","last_clean":"2024-04-26T18:22:59.433724+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:22:59.433724+0000","last_undegraded":"2024-04-26T18:22:59.433724+0000","last_fullsized":"2024-04-26T18:22:59.433724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":269,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:05.104003+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:23:05.104003+0000","last_peered":"2024-04-26T18:23:05.104003+0000","last_clean":"2024-04-26T18:23:05.104003+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:23:05.104003+0000","last_undegraded":"2024-04-26T18:23:05.104003+0000","last_fullsized":"2024-04-26T18:23:05.104003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":163,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010529+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:22:59.010529+0000","last_peered":"2024-04-26T18:22:59.010529+0000","last_clean":"2024-04-26T18:22:59.010529+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:22:59.010529+0000","last_undegraded":"2024-04-26T18:22:59.010529+0000","last_fullsized":"2024-04-26T18:22:59.010529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295100+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:22:59.295100+0000","last_peered":"2024-04-26T18:22:59.295100+0000","last_clean":"2024-04-26T18:22:59.295100+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:22:59.295100+0000","last_undegraded":"2024-04-26T18:22:59.295100+0000","last_fullsized":"2024-04-26T18:22:59.295100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295423+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:22:59.295423+0000","last_peered":"2024-04-26T18:22:59.295423+0000","last_clean":"2024-04-26T18:22:59.295423+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:22:59.295423+0000","last_undegraded":"2024-04-26T18:22:59.295423+0000","last_fullsized":"2024-04-26T18:22:59.295423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095205+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:22:59.095205+0000","last_peered":"2024-04-26T18:22:59.095205+0000","last_clean":"2024-04-26T18:22:59.095205+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:22:59.095205+0000","last_undegraded":"2024-04-26T18:22:59.095205+0000","last_fullsized":"2024-04-26T18:22:59.095205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089945+0000","last_change":"2024-04-26T18:21:58.049111+0000","last_active":"2024-04-26T18:22:59.089945+0000","last_peered":"2024-04-26T18:22:59.089945+0000","last_clean":"2024-04-26T18:22:59.089945+0000","last_became_active":"2024-04-26T18:21:58.048723+0000","last_became_peered":"2024-04-26T18:21:58.048723+0000","last_unstale":"2024-04-26T18:22:59.089945+0000","last_undegraded":"2024-04-26T18:22:59.089945+0000","last_fullsized":"2024-04-26T18:22:59.089945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:26.107528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:03.328115+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:23:03.328115+0000","last_peered":"2024-04-26T18:23:03.328115+0000","last_clean":"2024-04-26T18:23:03.328115+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:23:03.328115+0000","last_undegraded":"2024-04-26T18:23:03.328115+0000","last_fullsized":"2024-04-26T18:23:03.328115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089605+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:22:59.089605+0000","last_peered":"2024-04-26T18:22:59.089605+0000","last_clean":"2024-04-26T18:22:59.089605+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:22:59.089605+0000","last_undegraded":"2024-04-26T18:22:59.089605+0000","last_fullsized":"2024-04-26T18:22:59.089605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.027403+0000","last_change":"2024-04-26T18:22:59.027035+0000","last_active":"2024-04-26T18:22:59.027403+0000","last_peered":"2024-04-26T18:22:59.027403+0000","last_clean":"2024-04-26T18:22:59.027403+0000","last_became_active":"2024-04-26T18:22:59.015160+0000","last_became_peered":"2024-04-26T18:22:59.015160+0000","last_unstale":"2024-04-26T18:22:59.027403+0000","last_undegraded":"2024-04-26T18:22:59.027403+0000","last_fullsized":"2024-04-26T18:22:59.027403+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:04.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009669+0000","last_change":"2024-04-26T18:21:58.052169+0000","last_active":"2024-04-26T18:22:59.009669+0000","last_peered":"2024-04-26T18:22:59.009669+0000","last_clean":"2024-04-26T18:22:59.009669+0000","last_became_active":"2024-04-26T18:21:58.051756+0000","last_became_peered":"2024-04-26T18:21:58.051756+0000","last_unstale":"2024-04-26T18:22:59.009669+0000","last_undegraded":"2024-04-26T18:22:59.009669+0000","last_fullsized":"2024-04-26T18:22:59.009669+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:11.730323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010735+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:22:59.010735+0000","last_peered":"2024-04-26T18:22:59.010735+0000","last_clean":"2024-04-26T18:22:59.010735+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:22:59.010735+0000","last_undegraded":"2024-04-26T18:22:59.010735+0000","last_fullsized":"2024-04-26T18:22:59.010735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012271+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:22:59.012271+0000","last_peered":"2024-04-26T18:22:59.012271+0000","last_clean":"2024-04-26T18:22:59.012271+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:22:59.012271+0000","last_undegraded":"2024-04-26T18:22:59.012271+0000","last_fullsized":"2024-04-26T18:22:59.012271+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009818+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:22:59.009818+0000","last_peered":"2024-04-26T18:22:59.009818+0000","last_clean":"2024-04-26T18:22:59.009818+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:22:59.009818+0000","last_undegraded":"2024-04-26T18:22:59.009818+0000","last_fullsized":"2024-04-26T18:22:59.009818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.011128+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:22:59.011128+0000","last_peered":"2024-04-26T18:22:59.011128+0000","last_clean":"2024-04-26T18:22:59.011128+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:22:59.011128+0000","last_undegraded":"2024-04-26T18:22:59.011128+0000","last_fullsized":"2024-04-26T18:22:59.011128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089834+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:22:59.089834+0000","last_peered":"2024-04-26T18:22:59.089834+0000","last_clean":"2024-04-26T18:22:59.089834+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:22:59.089834+0000","last_undegraded":"2024-04-26T18:22:59.089834+0000","last_fullsized":"2024-04-26T18:22:59.089834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010827+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:22:59.010827+0000","last_peered":"2024-04-26T18:22:59.010827+0000","last_clean":"2024-04-26T18:22:59.010827+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:22:59.010827+0000","last_undegraded":"2024-04-26T18:22:59.010827+0000","last_fullsized":"2024-04-26T18:22:59.010827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095605+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:22:59.095605+0000","last_peered":"2024-04-26T18:22:59.095605+0000","last_clean":"2024-04-26T18:22:59.095605+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:22:59.095605+0000","last_undegraded":"2024-04-26T18:22:59.095605+0000","last_fullsized":"2024-04-26T18:22:59.095605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:23:04.329059+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:23:04.329059+0000","last_peered":"2024-04-26T18:23:04.329059+0000","last_clean":"2024-04-26T18:23:04.329059+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:23:04.329059+0000","last_undegraded":"2024-04-26T18:23:04.329059+0000","last_fullsized":"2024-04-26T18:23:04.329059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294859+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:22:59.294859+0000","last_peered":"2024-04-26T18:22:59.294859+0000","last_clean":"2024-04-26T18:22:59.294859+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:22:59.294859+0000","last_undegraded":"2024-04-26T18:22:59.294859+0000","last_fullsized":"2024-04-26T18:22:59.294859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014293+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:22:59.014293+0000","last_peered":"2024-04-26T18:22:59.014293+0000","last_clean":"2024-04-26T18:22:59.014293+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:22:59.014293+0000","last_undegraded":"2024-04-26T18:22:59.014293+0000","last_fullsized":"2024-04-26T18:22:59.014293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089701+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:22:59.089701+0000","last_peered":"2024-04-26T18:22:59.089701+0000","last_clean":"2024-04-26T18:22:59.089701+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:22:59.089701+0000","last_undegraded":"2024-04-26T18:22:59.089701+0000","last_fullsized":"2024-04-26T18:22:59.089701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294996+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:22:59.294996+0000","last_peered":"2024-04-26T18:22:59.294996+0000","last_clean":"2024-04-26T18:22:59.294996+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:22:59.294996+0000","last_undegraded":"2024-04-26T18:22:59.294996+0000","last_fullsized":"2024-04-26T18:22:59.294996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089835+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:22:59.089835+0000","last_peered":"2024-04-26T18:22:59.089835+0000","last_clean":"2024-04-26T18:22:59.089835+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:22:59.089835+0000","last_undegraded":"2024-04-26T18:22:59.089835+0000","last_fullsized":"2024-04-26T18:22:59.089835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010551+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:22:59.010551+0000","last_peered":"2024-04-26T18:22:59.010551+0000","last_clean":"2024-04-26T18:22:59.010551+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:22:59.010551+0000","last_undegraded":"2024-04-26T18:22:59.010551+0000","last_fullsized":"2024-04-26T18:22:59.010551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295541+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:22:59.295541+0000","last_peered":"2024-04-26T18:22:59.295541+0000","last_clean":"2024-04-26T18:22:59.295541+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:22:59.295541+0000","last_undegraded":"2024-04-26T18:22:59.295541+0000","last_fullsized":"2024-04-26T18:22:59.295541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009966+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:22:59.009966+0000","last_peered":"2024-04-26T18:22:59.009966+0000","last_clean":"2024-04-26T18:22:59.009966+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:22:59.009966+0000","last_undegraded":"2024-04-26T18:22:59.009966+0000","last_fullsized":"2024-04-26T18:22:59.009966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295245+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:22:59.295245+0000","last_peered":"2024-04-26T18:22:59.295245+0000","last_clean":"2024-04-26T18:22:59.295245+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:22:59.295245+0000","last_undegraded":"2024-04-26T18:22:59.295245+0000","last_fullsized":"2024-04-26T18:22:59.295245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014653+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:22:59.014653+0000","last_peered":"2024-04-26T18:22:59.014653+0000","last_clean":"2024-04-26T18:22:59.014653+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:22:59.014653+0000","last_undegraded":"2024-04-26T18:22:59.014653+0000","last_fullsized":"2024-04-26T18:22:59.014653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014363+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:22:59.014363+0000","last_peered":"2024-04-26T18:22:59.014363+0000","last_clean":"2024-04-26T18:22:59.014363+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:22:59.014363+0000","last_undegraded":"2024-04-26T18:22:59.014363+0000","last_fullsized":"2024-04-26T18:22:59.014363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433592+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:22:59.433592+0000","last_peered":"2024-04-26T18:22:59.433592+0000","last_clean":"2024-04-26T18:22:59.433592+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:22:59.433592+0000","last_undegraded":"2024-04-26T18:22:59.433592+0000","last_fullsized":"2024-04-26T18:22:59.433592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010291+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:22:59.010291+0000","last_peered":"2024-04-26T18:22:59.010291+0000","last_clean":"2024-04-26T18:22:59.010291+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:22:59.010291+0000","last_undegraded":"2024-04-26T18:22:59.010291+0000","last_fullsized":"2024-04-26T18:22:59.010291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294569+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:22:59.294569+0000","last_peered":"2024-04-26T18:22:59.294569+0000","last_clean":"2024-04-26T18:22:59.294569+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:22:59.294569+0000","last_undegraded":"2024-04-26T18:22:59.294569+0000","last_fullsized":"2024-04-26T18:22:59.294569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012832+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:22:59.012832+0000","last_peered":"2024-04-26T18:22:59.012832+0000","last_clean":"2024-04-26T18:22:59.012832+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:22:59.012832+0000","last_undegraded":"2024-04-26T18:22:59.012832+0000","last_fullsized":"2024-04-26T18:22:59.012832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012067+0000","last_change":"2024-04-26T18:21:58.057410+0000","last_active":"2024-04-26T18:22:59.012067+0000","last_peered":"2024-04-26T18:22:59.012067+0000","last_clean":"2024-04-26T18:22:59.012067+0000","last_became_active":"2024-04-26T18:21:58.054453+0000","last_became_peered":"2024-04-26T18:21:58.054453+0000","last_unstale":"2024-04-26T18:22:59.012067+0000","last_undegraded":"2024-04-26T18:22:59.012067+0000","last_fullsized":"2024-04-26T18:22:59.012067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:22.856834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294758+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:22:59.294758+0000","last_peered":"2024-04-26T18:22:59.294758+0000","last_clean":"2024-04-26T18:22:59.294758+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:22:59.294758+0000","last_undegraded":"2024-04-26T18:22:59.294758+0000","last_fullsized":"2024-04-26T18:22:59.294758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.010476+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:22:59.010476+0000","last_peered":"2024-04-26T18:22:59.010476+0000","last_clean":"2024-04-26T18:22:59.010476+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:22:59.010476+0000","last_undegraded":"2024-04-26T18:22:59.010476+0000","last_fullsized":"2024-04-26T18:22:59.010476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089868+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:22:59.089868+0000","last_peered":"2024-04-26T18:22:59.089868+0000","last_clean":"2024-04-26T18:22:59.089868+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:22:59.089868+0000","last_undegraded":"2024-04-26T18:22:59.089868+0000","last_fullsized":"2024-04-26T18:22:59.089868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.014487+0000","last_change":"2024-04-26T18:21:58.069561+0000","last_active":"2024-04-26T18:22:59.014487+0000","last_peered":"2024-04-26T18:22:59.014487+0000","last_clean":"2024-04-26T18:22:59.014487+0000","last_became_active":"2024-04-26T18:21:58.053199+0000","last_became_peered":"2024-04-26T18:21:58.053199+0000","last_unstale":"2024-04-26T18:22:59.014487+0000","last_undegraded":"2024-04-26T18:22:59.014487+0000","last_fullsized":"2024-04-26T18:22:59.014487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:27.166045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090184+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:22:59.090184+0000","last_peered":"2024-04-26T18:22:59.090184+0000","last_clean":"2024-04-26T18:22:59.090184+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:22:59.090184+0000","last_undegraded":"2024-04-26T18:22:59.090184+0000","last_fullsized":"2024-04-26T18:22:59.090184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095420+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:22:59.095420+0000","last_peered":"2024-04-26T18:22:59.095420+0000","last_clean":"2024-04-26T18:22:59.095420+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:22:59.095420+0000","last_undegraded":"2024-04-26T18:22:59.095420+0000","last_fullsized":"2024-04-26T18:22:59.095420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.089990+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:22:59.089990+0000","last_peered":"2024-04-26T18:22:59.089990+0000","last_clean":"2024-04-26T18:22:59.089990+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:22:59.089990+0000","last_undegraded":"2024-04-26T18:22:59.089990+0000","last_fullsized":"2024-04-26T18:22:59.089990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295216+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:22:59.295216+0000","last_peered":"2024-04-26T18:22:59.295216+0000","last_clean":"2024-04-26T18:22:59.295216+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:22:59.295216+0000","last_undegraded":"2024-04-26T18:22:59.295216+0000","last_fullsized":"2024-04-26T18:22:59.295216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.090164+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:22:59.090164+0000","last_peered":"2024-04-26T18:22:59.090164+0000","last_clean":"2024-04-26T18:22:59.090164+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:22:59.090164+0000","last_undegraded":"2024-04-26T18:22:59.090164+0000","last_fullsized":"2024-04-26T18:22:59.090164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012833+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:22:59.012833+0000","last_peered":"2024-04-26T18:22:59.012833+0000","last_clean":"2024-04-26T18:22:59.012833+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:22:59.012833+0000","last_undegraded":"2024-04-26T18:22:59.012833+0000","last_fullsized":"2024-04-26T18:22:59.012833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295045+0000","last_change":"2024-04-26T18:21:58.052815+0000","last_active":"2024-04-26T18:22:59.295045+0000","last_peered":"2024-04-26T18:22:59.295045+0000","last_clean":"2024-04-26T18:22:59.295045+0000","last_became_active":"2024-04-26T18:21:58.052405+0000","last_became_peered":"2024-04-26T18:21:58.052405+0000","last_unstale":"2024-04-26T18:22:59.295045+0000","last_undegraded":"2024-04-26T18:22:59.295045+0000","last_fullsized":"2024-04-26T18:22:59.295045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:29.289089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.294975+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:22:59.294975+0000","last_peered":"2024-04-26T18:22:59.294975+0000","last_clean":"2024-04-26T18:22:59.294975+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:22:59.294975+0000","last_undegraded":"2024-04-26T18:22:59.294975+0000","last_fullsized":"2024-04-26T18:22:59.294975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.013993+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:22:59.013993+0000","last_peered":"2024-04-26T18:22:59.013993+0000","last_clean":"2024-04-26T18:22:59.013993+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:22:59.013993+0000","last_undegraded":"2024-04-26T18:22:59.013993+0000","last_fullsized":"2024-04-26T18:22:59.013993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295554+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:22:59.295554+0000","last_peered":"2024-04-26T18:22:59.295554+0000","last_clean":"2024-04-26T18:22:59.295554+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:22:59.295554+0000","last_undegraded":"2024-04-26T18:22:59.295554+0000","last_fullsized":"2024-04-26T18:22:59.295554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012283+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:22:59.012283+0000","last_peered":"2024-04-26T18:22:59.012283+0000","last_clean":"2024-04-26T18:22:59.012283+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:22:59.012283+0000","last_undegraded":"2024-04-26T18:22:59.012283+0000","last_fullsized":"2024-04-26T18:22:59.012283+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295422+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:22:59.295422+0000","last_peered":"2024-04-26T18:22:59.295422+0000","last_clean":"2024-04-26T18:22:59.295422+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:22:59.295422+0000","last_undegraded":"2024-04-26T18:22:59.295422+0000","last_fullsized":"2024-04-26T18:22:59.295422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295137+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:22:59.295137+0000","last_peered":"2024-04-26T18:22:59.295137+0000","last_clean":"2024-04-26T18:22:59.295137+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:22:59.295137+0000","last_undegraded":"2024-04-26T18:22:59.295137+0000","last_fullsized":"2024-04-26T18:22:59.295137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.095207+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:22:59.095207+0000","last_peered":"2024-04-26T18:22:59.095207+0000","last_clean":"2024-04-26T18:22:59.095207+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:22:59.095207+0000","last_undegraded":"2024-04-26T18:22:59.095207+0000","last_fullsized":"2024-04-26T18:22:59.095207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.009525+0000","last_change":"2024-04-26T18:21:58.051597+0000","last_active":"2024-04-26T18:22:59.009525+0000","last_peered":"2024-04-26T18:22:59.009525+0000","last_clean":"2024-04-26T18:22:59.009525+0000","last_became_active":"2024-04-26T18:21:58.051139+0000","last_became_peered":"2024-04-26T18:21:58.051139+0000","last_unstale":"2024-04-26T18:22:59.009525+0000","last_undegraded":"2024-04-26T18:22:59.009525+0000","last_fullsized":"2024-04-26T18:22:59.009525+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.289734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.433776+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:22:59.433776+0000","last_peered":"2024-04-26T18:22:59.433776+0000","last_clean":"2024-04-26T18:22:59.433776+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:22:59.433776+0000","last_undegraded":"2024-04-26T18:22:59.433776+0000","last_fullsized":"2024-04-26T18:22:59.433776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.295406+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:22:59.295406+0000","last_peered":"2024-04-26T18:22:59.295406+0000","last_clean":"2024-04-26T18:22:59.295406+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:22:59.295406+0000","last_undegraded":"2024-04-26T18:22:59.295406+0000","last_fullsized":"2024-04-26T18:22:59.295406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:22:59.012125+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:22:59.012125+0000","last_peered":"2024-04-26T18:22:59.012125+0000","last_clean":"2024-04-26T18:22:59.012125+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:22:59.012125+0000","last_undegraded":"2024-04-26T18:22:59.012125+0000","last_fullsized":"2024-04-26T18:22:59.012125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":620,"num_read_kb":379,"num_write":171,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1276,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383720,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40508,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704644,"statfs":{"total":95995035648,"available":95953555456,"internally_reserved":0,"allocated":1028096,"data_stored":497190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1149155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":497314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":496540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332155,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1089305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":496703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":557674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":497974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40576,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704576,"statfs":{"total":95995035648,"available":95953485824,"internally_reserved":0,"allocated":1097728,"data_stored":558280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":497014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":497834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757318,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1088989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":44,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:23:10.984 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:23:10.984 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T18:23:10.984 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T18:23:10.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T18:23:11.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:11 smithi037 ceph-mon[29171]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:11.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:11 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:11.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:11 smithi170 ceph-mon[32697]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:11.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:11 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:11.336 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:11 smithi120 ceph-mon[32244]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:23:11.336 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:11 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:12.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:12 smithi037 ceph-mon[29171]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 437 B/s wr, 7 op/s 2024-04-26T18:23:12.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:12 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:12 smithi170 ceph-mon[32697]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 437 B/s wr, 7 op/s 2024-04-26T18:23:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:12 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:12 smithi120 ceph-mon[32244]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 437 B/s wr, 7 op/s 2024-04-26T18:23:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:12 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 -- 172.21.15.37:0/3355619554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 msgr2=0x7ff5fc110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 --2- 172.21.15.37:0/3355619554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc110390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b0d0 tx=0x7ff5f400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 -- 172.21.15.37:0/3355619554 shutdown_connections 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 --2- 172.21.15.37:0/3355619554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc110390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 --2- 172.21.15.37:0/3355619554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc1084b0 0x7ff5fc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 --2- 172.21.15.37:0/3355619554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 0x7ff5fc06bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.574+0000 7ff601730700 1 -- 172.21.15.37:0/3355619554 >> 172.21.15.37:0/3355619554 conn(0x7ff5fc079120 msgr2=0x7ff5fc079530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:13.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.575+0000 7ff601730700 1 -- 172.21.15.37:0/3355619554 shutdown_connections 2024-04-26T18:23:13.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.575+0000 7ff601730700 1 -- 172.21.15.37:0/3355619554 wait complete. 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.575+0000 7ff601730700 1 Processor -- start 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.575+0000 7ff601730700 1 -- start start 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 0x7ff5fc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc1084b0 0x7ff5fc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc113650 con 0x7ff5fc06c520 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1134d0 con 0x7ff5fc06bb60 2024-04-26T18:23:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff601730700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1137d0 con 0x7ff5fc1084b0 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49954/0 (socket says 172.21.15.37:49954) 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 -- 172.21.15.37:0/4192887006 learned_addr learned my addr 172.21.15.37:0/4192887006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5faffd700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 0x7ff5fc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 -- 172.21.15.37:0/4192887006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc1084b0 msgr2=0x7ff5fc1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc1084b0 0x7ff5fc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 -- 172.21.15.37:0/4192887006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 msgr2=0x7ff5fc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 0x7ff5fc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 -- 172.21.15.37:0/4192887006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19a050 con 0x7ff5fc06c520 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f3fff700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400d950 tx=0x7ff5e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.576+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4016070 con 0x7ff5fc06c520 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.577+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19a340 con 0x7ff5fc06c520 2024-04-26T18:23:13.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.577+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e4009df0 con 0x7ff5fc06c520 2024-04-26T18:23:13.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.577+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400f3c0 con 0x7ff5fc06c520 2024-04-26T18:23:13.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.577+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a6600 con 0x7ff5fc06c520 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.578+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc06a110 con 0x7ff5fc06c520 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.579+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff5e4019070 con 0x7ff5fc06c520 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.580+0000 7ff5f8ff9700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 0x7ff5e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.580+0000 7ff5faffd700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 0x7ff5e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.580+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7ff5e40d0000 con 0x7ff5fc06c520 2024-04-26T18:23:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.580+0000 7ff5faffd700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 0x7ff5e80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005950 tx=0x7ff5ec00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:13.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.585+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5e409c9b0 con 0x7ff5fc06c520 2024-04-26T18:23:13.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.890+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff5fc02e490 con 0x7ff5fc06c520 2024-04-26T18:23:13.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.891+0000 7ff5f8ff9700 1 -- 172.21.15.37:0/4192887006 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff5e4096e20 con 0x7ff5fc06c520 2024-04-26T18:23:13.891 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:13.891 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 msgr2=0x7ff5e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 0x7ff5e80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec005950 tx=0x7ff5ec00a300 comp rx=0 tx=0).stop 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 msgr2=0x7ff5fc19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400d950 tx=0x7ff5e400dc60 comp rx=0 tx=0).stop 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 shutdown_connections 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff5e80a3c40 0x7ff5e80a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5fc1084b0 0x7ff5fc1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff5fc06bb60 0x7ff5fc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 --2- 172.21.15.37:0/4192887006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5fc06c520 0x7ff5fc19fc70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:13.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 >> 172.21.15.37:0/4192887006 conn(0x7ff5fc079120 msgr2=0x7ff5fc1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:13.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 shutdown_connections 2024-04-26T18:23:13.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:13.894+0000 7ff601730700 1 -- 172.21.15.37:0/4192887006 wait complete. 2024-04-26T18:23:14.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:14 smithi037 ceph-mon[29171]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:14.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:14 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:14 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4192887006' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:23:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:14 smithi170 ceph-mon[32697]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:14 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:14 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4192887006' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:23:14.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:14 smithi120 ceph-mon[32244]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:14 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:14 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4192887006' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:23:14.604 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T18:23:14.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:14.873 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:15.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:15 smithi037 ceph-mon[29171]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:15.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:15 smithi170 ceph-mon[32697]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:15 smithi120 ceph-mon[32244]: from='mgr.14152 172.21.15.37:0/1883497091' entity='mgr.x' 2024-04-26T18:23:16.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:16 smithi037 ceph-mon[29171]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:16.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:16 smithi170 ceph-mon[32697]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:16 smithi120 ceph-mon[32244]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 -- 172.21.15.37:0/1302000060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 msgr2=0x7faabc1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 --2- 172.21.15.37:0/1302000060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc1098c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faaac009990 tx=0x7faaac009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 -- 172.21.15.37:0/1302000060 shutdown_connections 2024-04-26T18:23:16.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 --2- 172.21.15.37:0/1302000060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc103e60 0x7faabc1102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 --2- 172.21.15.37:0/1302000060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc1098c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 --2- 172.21.15.37:0/1302000060 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc1034a0 0x7faabc103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 -- 172.21.15.37:0/1302000060 >> 172.21.15.37:0/1302000060 conn(0x7faabc07a590 msgr2=0x7faabc07a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 -- 172.21.15.37:0/1302000060 shutdown_connections 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.558+0000 7faac1d1b700 1 -- 172.21.15.37:0/1302000060 wait complete. 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 Processor -- start 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 -- start start 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc1034a0 0x7faabc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc103e60 0x7faabc19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faabc1135e0 con 0x7faabc1094e0 2024-04-26T18:23:16.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faabc113460 con 0x7faabc103e60 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.559+0000 7faac1d1b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faabc113760 con 0x7faabc1034a0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc1034a0 0x7faabc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42316/0 (socket says 172.21.15.37:42316) 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 -- 172.21.15.37:0/464369633 learned_addr learned my addr 172.21.15.37:0/464369633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabaffd700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc103e60 0x7faabc19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 -- 172.21.15.37:0/464369633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc1034a0 msgr2=0x7faabc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc1034a0 0x7faabc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 -- 172.21.15.37:0/464369633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc103e60 msgr2=0x7faabc19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc103e60 0x7faabc19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 -- 172.21.15.37:0/464369633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc199fe0 con 0x7faabc1094e0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faabbfff700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faab4007d70 tx=0x7faab400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4016020 con 0x7faabc1094e0 2024-04-26T18:23:16.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab400ac90 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc19a2d0 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40115e0 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.560+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1a64e0 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.561+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc104b80 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.562+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faab4019020 con 0x7faabc1094e0 2024-04-26T18:23:16.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.563+0000 7faab8ff9700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 0x7faaa40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:16.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.563+0000 7faabb7fe700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 0x7faaa40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:16.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.563+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7faab40d02a0 con 0x7faabc1094e0 2024-04-26T18:23:16.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.563+0000 7faabb7fe700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 0x7faaa40a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaac0098e0 tx=0x7faaac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:16.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.568+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faab4020020 con 0x7faabc1094e0 2024-04-26T18:23:16.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.821+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faabc19ad30 con 0x7faabc1094e0 2024-04-26T18:23:16.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.822+0000 7faab8ff9700 1 -- 172.21.15.37:0/464369633 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7faab409cc50 con 0x7faabc1094e0 2024-04-26T18:23:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:16.823 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:23:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 msgr2=0x7faaa40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 0x7faaa40a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaac0098e0 tx=0x7faaac005960 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 msgr2=0x7faabc199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faab4007d70 tx=0x7faab400a340 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 shutdown_connections 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7faaa40a3cc0 0x7faaa40a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc1094e0 0x7faabc199800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faabc1034a0 0x7faabc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 --2- 172.21.15.37:0/464369633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faabc103e60 0x7faabc19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 >> 172.21.15.37:0/464369633 conn(0x7faabc07a590 msgr2=0x7faabc102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 shutdown_connections 2024-04-26T18:23:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:16.824+0000 7faac1d1b700 1 -- 172.21.15.37:0/464369633 wait complete. 2024-04-26T18:23:16.826 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:23:17.182 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:17 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/464369633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:17.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:17 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/464369633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:17.495 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:17 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/464369633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:17.745 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:18.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:18 smithi037 ceph-mon[29171]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:18.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:18 smithi170 ceph-mon[32697]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:18.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:18 smithi120 ceph-mon[32244]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3618114371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf41025b0 msgr2=0x7f8bf4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3618114371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf41025b0 0x7f8bf4102990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00099c0 tx=0x7f8bf0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3618114371 shutdown_connections 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3618114371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf4107640 0x7f8bf410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3618114371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf41025b0 0x7f8bf4102990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3618114371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 0x7f8bf4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3618114371 >> 172.21.15.37:0/3618114371 conn(0x7f8bf40792d0 msgr2=0x7f8bf40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3618114371 shutdown_connections 2024-04-26T18:23:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.441+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3618114371 wait complete. 2024-04-26T18:23:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 Processor -- start 2024-04-26T18:23:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 -- start start 2024-04-26T18:23:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf41025b0 0x7f8bf419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 0x7f8bf419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf41124d0 con 0x7f8bf4107640 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4112350 con 0x7f8bf41025b0 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.442+0000 7f8bfb6c6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4112650 con 0x7f8bf4102f60 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42346/0 (socket says 172.21.15.37:42346) 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 -- 172.21.15.37:0/3659001080 learned_addr learned my addr 172.21.15.37:0/3659001080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9462700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf41025b0 0x7f8bf419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf8c61700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 0x7f8bf419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 -- 172.21.15.37:0/3659001080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 msgr2=0x7f8bf419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 0x7f8bf419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 -- 172.21.15.37:0/3659001080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf41025b0 msgr2=0x7f8bf419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf41025b0 0x7f8bf419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 -- 172.21.15.37:0/3659001080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf419a100 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bf9c63700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bec007ee0 tx=0x7f8bec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004bb0 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bec004d50 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf419a3f0 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec005920 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.443+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41a6660 con 0x7f8bf4107640 2024-04-26T18:23:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.444+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf406a110 con 0x7f8bf4107640 2024-04-26T18:23:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.445+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8bec01a020 con 0x7f8bf4107640 2024-04-26T18:23:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.446+0000 7f8bea7fc700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 0x7f8be00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.446+0000 7f8bf9462700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 0x7f8be00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.446+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f8bec0d0020 con 0x7f8bf4107640 2024-04-26T18:23:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.447+0000 7f8bf9462700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 0x7f8be00a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.450+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bec010070 con 0x7f8bf4107640 2024-04-26T18:23:19.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.703+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bf402e490 con 0x7f8bf4107640 2024-04-26T18:23:19.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.704+0000 7f8bea7fc700 1 -- 172.21.15.37:0/3659001080 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7f8bec096e20 con 0x7f8bf4107640 2024-04-26T18:23:19.704 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:19.705 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:23:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 msgr2=0x7f8be00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:19.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 0x7f8be00a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf0005fb0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 msgr2=0x7f8bf41999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8bec007ee0 tx=0x7f8bec00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 shutdown_connections 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8be00a3cc0 0x7f8be00a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bf4107640 0x7f8bf41999a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bf41025b0 0x7f8bf419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 --2- 172.21.15.37:0/3659001080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bf4102f60 0x7f8bf419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 >> 172.21.15.37:0/3659001080 conn(0x7f8bf40792d0 msgr2=0x7f8bf41063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 shutdown_connections 2024-04-26T18:23:19.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:19.706+0000 7f8bfb6c6700 1 -- 172.21.15.37:0/3659001080 wait complete. 2024-04-26T18:23:19.708 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:23:20.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:20 smithi170 ceph-mon[32697]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:20.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:20 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3659001080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:20.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:20 smithi037 ceph-mon[29171]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:20.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:20 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3659001080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:20.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:23:20.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:20 smithi120 ceph-mon[32244]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:20.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:20 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3659001080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:20.663 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:22 smithi170 ceph-mon[32697]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:22.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 -- 172.21.15.37:0/1977739427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 msgr2=0x7fbd2c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 --2- 172.21.15.37:0/1977739427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c1038f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009990 tx=0x7fbd20009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 -- 172.21.15.37:0/1977739427 shutdown_connections 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 --2- 172.21.15.37:0/1977739427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c103e30 0x7fbd2c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 --2- 172.21.15.37:0/1977739427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 --2- 172.21.15.37:0/1977739427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c1038f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 -- 172.21.15.37:0/1977739427 >> 172.21.15.37:0/1977739427 conn(0x7fbd2c07a580 msgr2=0x7fbd2c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 -- 172.21.15.37:0/1977739427 shutdown_connections 2024-04-26T18:23:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 -- 172.21.15.37:0/1977739427 wait complete. 2024-04-26T18:23:22.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:22 smithi037 ceph-mon[29171]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.331+0000 7fbd33023700 1 Processor -- start 2024-04-26T18:23:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 -- start start 2024-04-26T18:23:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c103e30 0x7fbd2c10fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c113670 con 0x7fbd2c103470 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1134f0 con 0x7fbd2c103e30 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd33023700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1137f0 con 0x7fbd2c1094c0 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42370/0 (socket says 172.21.15.37:42370) 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 -- 172.21.15.37:0/2393469000 learned_addr learned my addr 172.21.15.37:0/2393469000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd2bfff700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c103e30 0x7fbd2c10fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd315c0700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 -- 172.21.15.37:0/2393469000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 msgr2=0x7fbd2c1a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 -- 172.21.15.37:0/2393469000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c103e30 msgr2=0x7fbd2c10fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c103e30 0x7fbd2c10fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd30dbf700 1 -- 172.21.15.37:0/2393469000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c1a52a0 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.332+0000 7fbd315c0700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd30dbf700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b530 tx=0x7fbd1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c015020 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c1a5590 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1c011ce0 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c010e40 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.333+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1ace10 con 0x7fbd2c103470 2024-04-26T18:23:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.334+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2c06a110 con 0x7fbd2c103470 2024-04-26T18:23:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.335+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbd1c018020 con 0x7fbd2c103470 2024-04-26T18:23:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.336+0000 7fbd29ffb700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 0x7fbd140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.336+0000 7fbd2bfff700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 0x7fbd140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.336+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7fbd1c0cfc80 con 0x7fbd2c103470 2024-04-26T18:23:22.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.337+0000 7fbd2bfff700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 0x7fbd140a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b5c0 tx=0x7fbd20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.341+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd1c01f020 con 0x7fbd2c103470 2024-04-26T18:23:22.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:22 smithi120 ceph-mon[32244]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.549+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd2c1a15f0 con 0x7fbd2c103470 2024-04-26T18:23:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.551+0000 7fbd29ffb700 1 -- 172.21.15.37:0/2393469000 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22725 (secure 0 0 0) 0x7fbd1c09c630 con 0x7fbd2c103470 2024-04-26T18:23:22.551 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:22.551 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":92,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:22:59.003338+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:23:22.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 msgr2=0x7fbd140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 0x7fbd140a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b5c0 tx=0x7fbd20005960 comp rx=0 tx=0).stop 2024-04-26T18:23:22.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 msgr2=0x7fbd2c10f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b530 tx=0x7fbd1c00b840 comp rx=0 tx=0).stop 2024-04-26T18:23:22.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 shutdown_connections 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbd140a3c40 0x7fbd140a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd2c1094c0 0x7fbd2c1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd2c103470 0x7fbd2c10f710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 --2- 172.21.15.37:0/2393469000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbd2c103e30 0x7fbd2c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 >> 172.21.15.37:0/2393469000 conn(0x7fbd2c07a580 msgr2=0x7fbd2c102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 shutdown_connections 2024-04-26T18:23:22.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:22.553+0000 7fbd33023700 1 -- 172.21.15.37:0/2393469000 wait complete. 2024-04-26T18:23:23.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:23.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:23 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2393469000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:23.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:23 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2393469000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:23.488 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:23 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2393469000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:24.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:24 smithi037 ceph-mon[29171]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:24.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:24 smithi170 ceph-mon[32697]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:24.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:24 smithi120 ceph-mon[32244]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T18:23:25.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3150533284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 msgr2=0x7fbbdc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3150533284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc1098d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00b0a0 tx=0x7fbbcc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3150533284 shutdown_connections 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3150533284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 0x7fbbdc1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3150533284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc1098d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3150533284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 0x7fbbdc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3150533284 >> 172.21.15.37:0/3150533284 conn(0x7fbbdc07a500 msgr2=0x7fbbdc07a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3150533284 shutdown_connections 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.171+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3150533284 wait complete. 2024-04-26T18:23:25.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 Processor -- start 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 -- start start 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 0x7fbbdc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 0x7fbbdc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc1136f0 con 0x7fbbdc1094f0 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc113570 con 0x7fbbdc103e90 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbe09fc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc113870 con 0x7fbbdc1034d0 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbdad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbdad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36732/0 (socket says 172.21.15.37:36732) 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbdad9d700 1 -- 172.21.15.37:0/3207196916 learned_addr learned my addr 172.21.15.37:0/3207196916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.172+0000 7fbbda59c700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 0x7fbbdc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbd9d9b700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 0x7fbbdc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 -- 172.21.15.37:0/3207196916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 msgr2=0x7fbbdc19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 0x7fbbdc19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 -- 172.21.15.37:0/3207196916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 msgr2=0x7fbbdc19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 0x7fbbdc19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 -- 172.21.15.37:0/3207196916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc19a1d0 con 0x7fbbdc1094f0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbdad9d700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b550 tx=0x7fbbd400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4015020 con 0x7fbbdc1094f0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd4011ce0 con 0x7fbbdc1094f0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc19a4c0 con 0x7fbbdc1094f0 2024-04-26T18:23:25.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4010c90 con 0x7fbbdc1094f0 2024-04-26T18:23:25.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.173+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc1a6780 con 0x7fbbdc1094f0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.174+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc06a110 con 0x7fbbdc1094f0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.175+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbbd4018020 con 0x7fbbdc1094f0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.176+0000 7fbbd37fe700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 0x7fbbc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.176+0000 7fbbda59c700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 0x7fbbc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.176+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7fbbd40cfae0 con 0x7fbbdc1094f0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.176+0000 7fbbda59c700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 0x7fbbc80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:25.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.181+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbd401f020 con 0x7fbbdc1094f0 2024-04-26T18:23:25.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.437+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbdc02e490 con 0x7fbbdc1094f0 2024-04-26T18:23:25.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.438+0000 7fbbd37fe700 1 -- 172.21.15.37:0/3207196916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7fbbd409c490 con 0x7fbbdc1094f0 2024-04-26T18:23:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:25.439 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.440+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 msgr2=0x7fbbc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.440+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 0x7fbbc80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc00b7b0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.440+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 msgr2=0x7fbbdc199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.440+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400b550 tx=0x7fbbd400b860 comp rx=0 tx=0).stop 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.440+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 shutdown_connections 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fbbc80a3c40 0x7fbbc80a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbdc1094f0 0x7fbbdc199a70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbdc1034d0 0x7fbbdc19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 --2- 172.21.15.37:0/3207196916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbdc103e90 0x7fbbdc19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 >> 172.21.15.37:0/3207196916 conn(0x7fbbdc07a500 msgr2=0x7fbbdc1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 shutdown_connections 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:25.441+0000 7fbbe09fc700 1 -- 172.21.15.37:0/3207196916 wait complete. 2024-04-26T18:23:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:23:26.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:21:10.996859+0000', 'modified': '2024-04-26T18:21:29.674118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24464, 'mds_1': 24449, 'mds_2': 24442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/707687219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 707687219}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 707687219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/3605086601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3605086601}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 3605086601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/2099915678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2099915678}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2099915678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6839/1569501885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1569501885}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1569501885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.120:6837/1266100832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1266100832}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1266100832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.120:6839/1246417570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1246417570}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1246417570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24464, 24442, 24449]} 2024-04-26T18:23:26.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:23:26.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:23:26.383 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:26.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:26 smithi037 ceph-mon[29171]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:26.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:26 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3207196916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:26.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:26 smithi120 ceph-mon[32244]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:26 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3207196916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:26.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:26 smithi170 ceph-mon[32697]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:26.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:26 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3207196916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 -- 172.21.15.37:0/1516293969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701094f0 msgr2=0x7f3d701098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1516293969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701094f0 0x7f3d701098d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009990 tx=0x7f3d6c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 -- 172.21.15.37:0/1516293969 shutdown_connections 2024-04-26T18:23:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1516293969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 0x7f3d701102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1516293969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701094f0 0x7f3d701098d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1516293969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701034d0 0x7f3d70103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 -- 172.21.15.37:0/1516293969 >> 172.21.15.37:0/1516293969 conn(0x7f3d7007a500 msgr2=0x7f3d7007a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 -- 172.21.15.37:0/1516293969 shutdown_connections 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.018+0000 7f3d77b64700 1 -- 172.21.15.37:0/1516293969 wait complete. 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 Processor -- start 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 -- start start 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 0x7f3d7019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701094f0 0x7f3d70199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d701136f0 con 0x7f3d701034d0 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d70113570 con 0x7f3d70103e90 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d77b64700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3d70113870 con 0x7f3d701094f0 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36744/0 (socket says 172.21.15.37:36744) 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 -- 172.21.15.37:0/1405637349 learned_addr learned my addr 172.21.15.37:0/1405637349 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 0x7f3d7019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d76101700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701094f0 0x7f3d70199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 -- 172.21.15.37:0/1405637349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701094f0 msgr2=0x7f3d70199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701094f0 0x7f3d70199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 -- 172.21.15.37:0/1405637349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 msgr2=0x7f3d7019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 0x7f3d7019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.019+0000 7f3d75900700 1 -- 172.21.15.37:0/1405637349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7019a1d0 con 0x7f3d701034d0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d75900700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c000c00 tx=0x7f3d6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d62ffd700 1 -- 172.21.15.37:0/1405637349 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c027070 con 0x7f3d701034d0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7019a460 con 0x7f3d701034d0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d62ffd700 1 -- 172.21.15.37:0/1405637349 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d6c0040e0 con 0x7f3d701034d0 2024-04-26T18:23:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d62ffd700 1 -- 172.21.15.37:0/1405637349 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00f460 con 0x7f3d701034d0 2024-04-26T18:23:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.020+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701a6780 con 0x7f3d701034d0 2024-04-26T18:23:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.022+0000 7f3d62ffd700 1 -- 172.21.15.37:0/1405637349 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d6c02a070 con 0x7f3d701034d0 2024-04-26T18:23:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.023+0000 7f3d62ffd700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 0x7f3d5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.023+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 0x7f3d5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.023+0000 7f3d62ffd700 1 -- 172.21.15.37:0/1405637349 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f3d6c0d1420 con 0x7f3d701034d0 2024-04-26T18:23:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.023+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 0x7f3d5c0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d7007acb0 tx=0x7f3d64009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.023+0000 7f3d823f6b80 1 Processor -- start 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 -- start start 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee31981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192160 0x55bee31925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bee30e07e0 con 0x55bee3192b20 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bee30468e0 con 0x55bee3197dc0 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bee30dc7a0 con 0x55bee3192160 2024-04-26T18:23:28.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36746/0 (socket says 172.21.15.37:36746) 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d75900700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee31981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 -- 172.21.15.37:0/3004122673 learned_addr learned my addr 172.21.15.37:0/3004122673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d750ff700 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192160 0x55bee31925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 -- 172.21.15.37:0/3004122673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192160 msgr2=0x55bee31925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192160 0x55bee31925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 -- 172.21.15.37:0/3004122673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 msgr2=0x55bee31981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee31981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d76101700 1 -- 172.21.15.37:0/3004122673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee302d880 con 0x55bee3192b20 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.024+0000 7f3d75900700 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee31981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.025+0000 7f3d76101700 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009f50 tx=0x7f3d68009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbae1cd3d9179efc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.025+0000 7f3d5bfff700 1 -- 172.21.15.37:0/3004122673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6801a070 con 0x55bee3192b20 2024-04-26T18:23:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.025+0000 7f3d5bfff700 1 -- 172.21.15.37:0/3004122673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d68015410 con 0x55bee3192b20 2024-04-26T18:23:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.025+0000 7f3d5bfff700 1 -- 172.21.15.37:0/3004122673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800a660 con 0x55bee3192b20 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- 172.21.15.37:0/3004122673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 msgr2=0x55bee319eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009f50 tx=0x7f3d68009f80 comp rx=0 tx=0).stop 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- 172.21.15.37:0/3004122673 shutdown_connections 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192b20 0x55bee319eb40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee31981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/3004122673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192160 0x55bee31925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- 172.21.15.37:0/3004122673 >> 172.21.15.37:0/3004122673 conn(0x55bee30bc570 msgr2=0x55bee3188900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- 172.21.15.37:0/3004122673 shutdown_connections 2024-04-26T18:23:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- 172.21.15.37:0/3004122673 wait complete. 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 -- start start 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.026+0000 7f3d823f6b80 1 Processor -- start 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee319bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bee319f0e0 con 0x55bee3192160 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bee309aa30 con 0x55bee3197dc0 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bee319f260 con 0x55bee3192b20 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d823f6b80 10 client.0 _ll_register_callbacks cb 0x55bee3078900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36750/0 (socket says 172.21.15.37:36750) 2024-04-26T18:23:28.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 -- 172.21.15.37:0/2458585265 learned_addr learned my addr 172.21.15.37:0/2458585265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d750ff700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d76101700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee319bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 -- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 msgr2=0x55bee319ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 -- 172.21.15.37:0/2458585265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 msgr2=0x55bee319bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee319bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 -- 172.21.15.37:0/2458585265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee309e430 con 0x55bee3192160 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d75900700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c09a130 tx=0x7f3d6c09d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e747d62e9d74f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d597fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:28.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.027+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b3060 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d68015410 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f3d6c09d9c0 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d6c09d0c0 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 10 client.14946 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bee31a9820 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d68015410 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3d6c0a5eb0 con 0x55bee3192160 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 10 client.14946 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 20 client.14946 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee31abf90 0x55bee31ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 4 client.14946 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:28.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.028+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bee30ee1c0 con 0x55bee31abf90 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.029+0000 7f3d750ff700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee31abf90 0x55bee31ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.029+0000 7f3d750ff700 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee31abf90 0x55bee31ad400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be7b2ffe1937ebf server_cookie=bab99a6af3aeb40 in_seq=0 out_seq=0 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.029+0000 7f3d597fa700 10 client.14946 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d597fa700 1 -- 172.21.15.37:0/2458585265 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bee30ee1c0 con 0x55bee31abf90 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d597fa700 10 client.14946 handle_command_reply: tid=0 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 client.14946 shutdown 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 msgr2=0x55bee319e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c09a130 tx=0x7f3d6c09d5b0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 shutdown_connections 2024-04-26T18:23:28.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee31abf90 0x55bee31ad400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3197dc0 0x55bee319bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3192160 0x55bee319e680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/2458585265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d597fa700 0 client.14946 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 >> 172.21.15.37:0/2458585265 conn(0x55bee30bc570 msgr2=0x55bee318cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 shutdown_connections 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 1 -- 172.21.15.37:0/2458585265 wait complete. 2024-04-26T18:23:28.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.030+0000 7f3d823f6b80 20 client.14946 trim_cache size 0 max 16384 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 Processor -- start 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 -- start start 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee31b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bee31ab740 con 0x55bee3197dc0 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bee31abe10 con 0x55bee3192160 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bee31ac4e0 con 0x55bee3192b20 2024-04-26T18:23:28.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36764/0 (socket says 172.21.15.37:36764) 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 -- 172.21.15.37:0/1654967331 learned_addr learned my addr 172.21.15.37:0/1654967331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee31b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 -- 172.21.15.37:0/1654967331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 msgr2=0x55bee31b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee31b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 -- 172.21.15.37:0/1654967331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 msgr2=0x55bee319b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 -- 172.21.15.37:0/1654967331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee31884d0 con 0x55bee3197dc0 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee31b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.051+0000 7f3d76101700 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009520 tx=0x7f3d68004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c662c1bae1b1cb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d5b7fe700 1 -- 172.21.15.37:0/1654967331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6801a040 con 0x55bee3197dc0 2024-04-26T18:23:28.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d5b7fe700 1 -- 172.21.15.37:0/1654967331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d68015410 con 0x55bee3197dc0 2024-04-26T18:23:28.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d5b7fe700 1 -- 172.21.15.37:0/1654967331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68020680 con 0x55bee3197dc0 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1654967331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 msgr2=0x55bee31b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009520 tx=0x7f3d68004be0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1654967331 shutdown_connections 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b6da0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1654967331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee31b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1654967331 >> 172.21.15.37:0/1654967331 conn(0x55bee30bc570 msgr2=0x55bee3188730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1654967331 shutdown_connections 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.052+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1654967331 wait complete. 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 -- start start 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 Processor -- start 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bee31ad280 con 0x55bee3197dc0 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bee31acbb0 con 0x55bee3192160 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bee31ad950 con 0x55bee3192b20 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d823f6b80 10 client.0 _ll_register_callbacks cb 0x55bee3078900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d76101700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36766/0 (socket says 172.21.15.37:36766) 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d76101700 1 -- 172.21.15.37:0/1780000381 learned_addr learned my addr 172.21.15.37:0/1780000381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.053+0000 7f3d750ff700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d75900700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 -- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 msgr2=0x55bee319d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 -- 172.21.15.37:0/1780000381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 msgr2=0x55bee319d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 -- 172.21.15.37:0/1780000381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee31b55e0 con 0x55bee3197dc0 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d76101700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d680095a0 tx=0x7f3d68009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535749eac20ffb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d597fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:28.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6802e060 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f3d680157d0 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d68015410 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d6801a040 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d823f6b80 10 client.14952 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bee3213980 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.054+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d68015410 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3d6801d030 con 0x55bee3197dc0 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 10 client.14952 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 10 client.14952 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 20 client.14952 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee3217e00 0x55bee3216b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 4 client.14952 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:23:28.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "status"}) -- 0x55bee30e8260 con 0x55bee3217e00 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d75900700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee3217e00 0x55bee3216b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d75900700 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee3217e00 0x55bee3216b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c37272e40fe33ae0 server_cookie=2ee3e4e32c010603 in_seq=0 out_seq=0 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d597fa700 10 client.14952 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d597fa700 1 -- 172.21.15.37:0/1780000381 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bee30e8260 con 0x55bee3217e00 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d597fa700 10 client.14952 handle_command_reply: tid=0 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.055+0000 7f3d823f6b80 1 client.14952 shutdown 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 msgr2=0x55bee31b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d680095a0 tx=0x7f3d68009a50 comp rx=0 tx=0).stop 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 shutdown_connections 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bee3197dc0 0x55bee31b4f70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55bee3217e00 0x55bee3216b10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bee3192160 0x55bee319d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 --2- 172.21.15.37:0/1780000381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bee3192b20 0x55bee319d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d597fa700 0 client.14952 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 >> 172.21.15.37:0/1780000381 conn(0x55bee30bc570 msgr2=0x55bee31886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 shutdown_connections 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 1 -- 172.21.15.37:0/1780000381 wait complete. 2024-04-26T18:23:28.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d823f6b80 20 client.14952 trim_cache size 0 max 16384 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24442, 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 123.11494826400001, 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:23:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:23:28.065 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 163.17341456400001 2024-04-26T18:23:28.065 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:23:28.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 msgr2=0x7f3d5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 0x7f3d5c0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d7007acb0 tx=0x7f3d64009190 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 msgr2=0x7f3d7019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.056+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c000c00 tx=0x7f3d6c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 shutdown_connections 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3d5c0a3c40 0x7f3d5c0a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d701094f0 0x7f3d70199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d701034d0 0x7f3d7019f8b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 --2- 172.21.15.37:0/1405637349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d70103e90 0x7f3d7019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 >> 172.21.15.37:0/1405637349 conn(0x7f3d7007a500 msgr2=0x7f3d70102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 shutdown_connections 2024-04-26T18:23:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:28.057+0000 7f3d77b64700 1 -- 172.21.15.37:0/1405637349 wait complete. 2024-04-26T18:23:28.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:28 smithi037 ceph-mon[29171]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:28 smithi120 ceph-mon[32244]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:28 smithi170 ceph-mon[32697]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:28.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:23:29.054 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:30.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:30 smithi037 ceph-mon[29171]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:30.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:30 smithi120 ceph-mon[32244]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:30.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:30 smithi170 ceph-mon[32697]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- 172.21.15.37:0/1964109629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88181094a0 msgr2=0x7f8818109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 --2- 172.21.15.37:0/1964109629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88181094a0 0x7f8818109880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8808009990 tx=0x7f8808009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- 172.21.15.37:0/1964109629 shutdown_connections 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 --2- 172.21.15.37:0/1964109629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806bad0 0x7f88181102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 --2- 172.21.15.37:0/1964109629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88181094a0 0x7f8818109880 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 --2- 172.21.15.37:0/1964109629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f881806b130 0x7f881806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- 172.21.15.37:0/1964109629 >> 172.21.15.37:0/1964109629 conn(0x7f88180792d0 msgr2=0x7f88180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- 172.21.15.37:0/1964109629 shutdown_connections 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- 172.21.15.37:0/1964109629 wait complete. 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 Processor -- start 2024-04-26T18:23:30.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.674+0000 7f881d656700 1 -- start start 2024-04-26T18:23:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806b130 0x7f88181a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88181094a0 0x7f881819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113590 con 0x7f881806bad0 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113410 con 0x7f881806b130 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8818113710 con 0x7f88181094a0 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36788/0 (socket says 172.21.15.37:36788) 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 -- 172.21.15.37:0/3297893939 learned_addr learned my addr 172.21.15.37:0/3297893939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88177fe700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88181094a0 0x7f881819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f8816ffd700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806b130 0x7f88181a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 -- 172.21.15.37:0/3297893939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88181094a0 msgr2=0x7f881819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88181094a0 0x7f881819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 -- 172.21.15.37:0/3297893939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806b130 msgr2=0x7f88181a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806b130 0x7f88181a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 -- 172.21.15.37:0/3297893939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819c190 con 0x7f881806bad0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f88167fc700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c9b0 tx=0x7f880c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f87fffff700 1 -- 172.21.15.37:0/3297893939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c007b00 con 0x7f881806bad0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881819c480 con 0x7f881806bad0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.675+0000 7f87fffff700 1 -- 172.21.15.37:0/3297893939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880c007ca0 con 0x7f881806bad0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.676+0000 7f87fffff700 1 -- 172.21.15.37:0/3297893939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c018660 con 0x7f881806bad0 2024-04-26T18:23:30.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.676+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a86f0 con 0x7f881806bad0 2024-04-26T18:23:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.678+0000 7f87fffff700 1 -- 172.21.15.37:0/3297893939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f880c00f450 con 0x7f881806bad0 2024-04-26T18:23:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.678+0000 7f87fffff700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 0x7f88000a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.679+0000 7f8816ffd700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 0x7f88000a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.679+0000 7f87fffff700 1 -- 172.21.15.37:0/3297893939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f880c0d0440 con 0x7f881806bad0 2024-04-26T18:23:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.679+0000 7f8816ffd700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 0x7f88000a5fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f880800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 Processor -- start 2024-04-26T18:23:30.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 -- start start 2024-04-26T18:23:30.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 0x5644402c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644402197e0 con 0x5644402cca20 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56444017f8e0 con 0x5644402c76f0 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644402157a0 con 0x5644402c6d50 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36798/0 (socket says 172.21.15.37:36798) 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.680+0000 7f8816ffd700 1 -- 172.21.15.37:0/433575767 learned_addr learned my addr 172.21.15.37:0/433575767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f88167fc700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 0x5644402c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f88177fe700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 -- 172.21.15.37:0/433575767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 msgr2=0x5644402c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 0x5644402c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 -- 172.21.15.37:0/433575767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 msgr2=0x5644402d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 -- 172.21.15.37:0/433575767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564440166880 con 0x5644402cca20 2024-04-26T18:23:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f88167fc700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 0x5644402c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f8816ffd700 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8808015000 tx=0x7f880801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696173365421a3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f87fd7fa700 1 -- 172.21.15.37:0/433575767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800f460 con 0x5644402cca20 2024-04-26T18:23:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f87fd7fa700 1 -- 172.21.15.37:0/433575767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880800fab0 con 0x5644402cca20 2024-04-26T18:23:30.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.681+0000 7f87fd7fa700 1 -- 172.21.15.37:0/433575767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808024ac0 con 0x5644402cca20 2024-04-26T18:23:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- 172.21.15.37:0/433575767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 msgr2=0x5644402cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8808015000 tx=0x7f880801f000 comp rx=0 tx=0).stop 2024-04-26T18:23:30.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- 172.21.15.37:0/433575767 shutdown_connections 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402cca20 0x5644402cce00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/433575767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c6d50 0x5644402c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- 172.21.15.37:0/433575767 >> 172.21.15.37:0/433575767 conn(0x5644401f5570 msgr2=0x5644402c1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- 172.21.15.37:0/433575767 shutdown_connections 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- 172.21.15.37:0/433575767 wait complete. 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 -- start start 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.682+0000 7f8827ee8b80 1 Processor -- start 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402cca20 0x5644402d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644402d3cb0 con 0x5644402c6d50 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644401d3a30 con 0x5644402c76f0 2024-04-26T18:23:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644402d3e30 con 0x5644402cca20 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 10 client.0 _ll_register_callbacks cb 0x5644401b1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36812/0 (socket says 172.21.15.37:36812) 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 -- 172.21.15.37:0/1211052869 learned_addr learned my addr 172.21.15.37:0/1211052869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f88167fc700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f88177fe700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402cca20 0x5644402d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 -- 172.21.15.37:0/1211052869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402cca20 msgr2=0x5644402d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402cca20 0x5644402d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 -- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 msgr2=0x5644402d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 -- 172.21.15.37:0/1211052869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644401d7430 con 0x5644402c6d50 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8816ffd700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8808031040 tx=0x7f880800ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd1d888f4780b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f87faff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803c060 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880800fab0 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.683+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f8808024990 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88080244e0 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 10 client.14964 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644402de3f0 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880800fab0 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f880802fbe0 con 0x5644402c6d50 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 10 client.14964 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 20 client.14964 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x5644402df740 0x5644402e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 4 client.14964 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644402271c0 con 0x5644402df740 2024-04-26T18:23:30.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.684+0000 7f88167fc700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x5644402df740 0x5644402e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.685+0000 7f87faff5700 10 client.14964 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.685+0000 7f88167fc700 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x5644402df740 0x5644402e18c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4607fbea56506 server_cookie=234ab50cee3e629c in_seq=0 out_seq=0 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f87faff5700 1 -- 172.21.15.37:0/1211052869 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644402271c0 con 0x5644402df740 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f87faff5700 10 client.14964 handle_command_reply: tid=0 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 client.14964 shutdown 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 msgr2=0x5644402d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8808031040 tx=0x7f880800ff90 comp rx=0 tx=0).stop 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 shutdown_connections 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x5644402df740 0x5644402e18c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402cca20 0x5644402d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402d3360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/1211052869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402c76f0 0x5644402d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 >> 172.21.15.37:0/1211052869 conn(0x5644401f5570 msgr2=0x5644402c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f87faff5700 0 client.14964 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:23:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 shutdown_connections 2024-04-26T18:23:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 1 -- 172.21.15.37:0/1211052869 wait complete. 2024-04-26T18:23:30.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.686+0000 7f8827ee8b80 20 client.14964 trim_cache size 0 max 16384 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 Processor -- start 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 -- start start 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644402dfc00 con 0x5644402c6d50 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644402e02d0 con 0x5644402cca20 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644402e09a0 con 0x5644402c76f0 2024-04-26T18:23:30.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36828/0 (socket says 172.21.15.37:36828) 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 -- 172.21.15.37:0/4218314641 learned_addr learned my addr 172.21.15.37:0/4218314641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f88177fe700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f88167fc700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 -- 172.21.15.37:0/4218314641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 msgr2=0x5644402bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 -- 172.21.15.37:0/4218314641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 msgr2=0x5644402c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f8816ffd700 1 -- 172.21.15.37:0/4218314641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644402d2fa0 con 0x5644402c6d50 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.707+0000 7f88177fe700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f8816ffd700 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880800ba90 tx=0x7f880800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713b837e2f95884 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f87fcff9700 1 -- 172.21.15.37:0/4218314641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808046020 con 0x5644402c6d50 2024-04-26T18:23:30.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f87fcff9700 1 -- 172.21.15.37:0/4218314641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8808024e40 con 0x5644402c6d50 2024-04-26T18:23:30.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f87fcff9700 1 -- 172.21.15.37:0/4218314641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803c040 con 0x5644402c6d50 2024-04-26T18:23:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f8827ee8b80 1 -- 172.21.15.37:0/4218314641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 msgr2=0x5644402c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.708+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880800ba90 tx=0x7f880800be70 comp rx=0 tx=0).stop 2024-04-26T18:23:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- 172.21.15.37:0/4218314641 shutdown_connections 2024-04-26T18:23:30.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402c7130 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/4218314641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- 172.21.15.37:0/4218314641 >> 172.21.15.37:0/4218314641 conn(0x5644401f5570 msgr2=0x5644402ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- 172.21.15.37:0/4218314641 shutdown_connections 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- 172.21.15.37:0/4218314641 wait complete. 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- start start 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 Processor -- start 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5644402e1740 con 0x5644402c6d50 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644402e1070 con 0x5644402cca20 2024-04-26T18:23:30.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644402e1e10 con 0x5644402c76f0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8827ee8b80 10 client.0 _ll_register_callbacks cb 0x5644401b1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8816ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36836/0 (socket says 172.21.15.37:36836) 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f8816ffd700 1 -- 172.21.15.37:0/2980682270 learned_addr learned my addr 172.21.15.37:0/2980682270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.709+0000 7f88167fc700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f88177fe700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 -- 172.21.15.37:0/2980682270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 msgr2=0x5644402e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 -- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 msgr2=0x5644402e8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402e8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 -- 172.21.15.37:0/2980682270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644402e9370 con 0x5644402c6d50 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f88167fc700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f88177fe700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8816ffd700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8808017460 tx=0x7f8808024c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523726b57cc16be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f87faff5700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:30.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808046030 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f88080179e0 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564440221260 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f880803c040 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8827ee8b80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644402ea710 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.710+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564440221260 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8808054070 con 0x5644402c6d50 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 20 client.14970 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x56444034ce00 0x56444034bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 4 client.14970 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "status"}) -- 0x564440099200 con 0x56444034ce00 2024-04-26T18:23:30.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f88167fc700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x56444034ce00 0x56444034bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f88167fc700 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x56444034ce00 0x56444034bb10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f00f1b209f434145 server_cookie=b349b6ad8bcc5bbf in_seq=0 out_seq=0 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.711+0000 7f87faff5700 10 client.14970 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f87faff5700 1 -- 172.21.15.37:0/2980682270 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564440099200 con 0x56444034ce00 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f87faff5700 10 client.14970 handle_command_reply: tid=0 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 client.14970 shutdown 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 msgr2=0x5644402e3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8808017460 tx=0x7f8808024c70 comp rx=0 tx=0).stop 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 shutdown_connections 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x56444034ce00 0x56444034bb10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5644402cca20 0x5644402e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5644402c6d50 0x5644402e3ec0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 --2- 172.21.15.37:0/2980682270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644402c76f0 0x5644402e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 >> 172.21.15.37:0/2980682270 conn(0x5644401f5570 msgr2=0x5644402c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f87faff5700 0 client.14970 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 shutdown_connections 2024-04-26T18:23:30.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 1 -- 172.21.15.37:0/2980682270 wait complete. 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.712+0000 7f8827ee8b80 20 client.14970 trim_cache size 0 max 16384 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 126.78204059700001, 2024-04-26T18:23:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 168.41762065200001 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 msgr2=0x7f88000a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 0x7f88000a5fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f880800b540 comp rx=0 tx=0).stop 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 msgr2=0x7f88181a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:30.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c9b0 tx=0x7f880c00ccc0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 shutdown_connections 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f88000a3b20 0x7f88000a5fe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88181094a0 0x7f881819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881806b130 0x7f88181a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 --2- 172.21.15.37:0/3297893939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f881806bad0 0x7f88181a1d60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 >> 172.21.15.37:0/3297893939 conn(0x7f88180792d0 msgr2=0x7f881810e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 shutdown_connections 2024-04-26T18:23:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:30.713+0000 7f881d656700 1 -- 172.21.15.37:0/3297893939 wait complete. 2024-04-26T18:23:31.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:23:31.655 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:32.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:32 smithi037 ceph-mon[29171]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:32.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:32 smithi120 ceph-mon[32244]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:32 smithi170 ceph-mon[32697]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:33.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2769573047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 msgr2=0x7f5b94110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2769573047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b94110310 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00b0a0 tx=0x7f5b8c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2769573047 shutdown_connections 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2769573047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b94110310 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2769573047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b941094d0 0x7f5b941098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2769573047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b9406bbd0 0x7f5b9406c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2769573047 >> 172.21.15.37:0/2769573047 conn(0x7f5b940793c0 msgr2=0x7f5b940797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2769573047 shutdown_connections 2024-04-26T18:23:33.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.412+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2769573047 wait complete. 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 Processor -- start 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 -- start start 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 0x7f5b94102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b941094d0 0x7f5b94102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94113660 con 0x7f5b9406c590 2024-04-26T18:23:33.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941134e0 con 0x7f5b941094d0 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b99a3e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941137e0 con 0x7f5b9406bbd0 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36854/0 (socket says 172.21.15.37:36854) 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 -- 172.21.15.37:0/2693586463 learned_addr learned my addr 172.21.15.37:0/2693586463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b937fe700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b941094d0 0x7f5b94102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 0x7f5b94102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 -- 172.21.15.37:0/2693586463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 msgr2=0x7f5b94102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 0x7f5b94102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.413+0000 7f5b927fc700 1 -- 172.21.15.37:0/2693586463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b941094d0 msgr2=0x7f5b94102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b941094d0 0x7f5b94102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b927fc700 1 -- 172.21.15.37:0/2693586463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b941a7f80 con 0x7f5b9406c590 2024-04-26T18:23:33.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 0x7f5b94102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c9b0 tx=0x7f5b8800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b98a3c700 1 -- 172.21.15.37:0/2693586463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007b00 con 0x7f5b9406c590 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b98a3c700 1 -- 172.21.15.37:0/2693586463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b88007ca0 con 0x7f5b9406c590 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b941a8270 con 0x7f5b9406c590 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b98a3c700 1 -- 172.21.15.37:0/2693586463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88018660 con 0x7f5b9406c590 2024-04-26T18:23:33.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.414+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a87b0 con 0x7f5b9406c590 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.416+0000 7f5b98a3c700 1 -- 172.21.15.37:0/2693586463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b8800f450 con 0x7f5b9406c590 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.417+0000 7f5b98a3c700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 0x7f5b7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.417+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 0x7f5b7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.417+0000 7f5b98a3c700 1 -- 172.21.15.37:0/2693586463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f5b880d0450 con 0x7f5b9406c590 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.417+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 0x7f5b7c0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840099d0 tx=0x7f5b84008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 Processor -- start 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 -- start start 2024-04-26T18:23:33.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5f1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703e5377e0 con 0x55703e5e4d50 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55703e49d8e0 con 0x55703e5e56f0 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55703e5337a0 con 0x55703e5eaa20 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36860/0 (socket says 172.21.15.37:36860) 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 -- 172.21.15.37:0/2276845641 learned_addr learned my addr 172.21.15.37:0/2276845641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b937fe700 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 -- 172.21.15.37:0/2276845641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 msgr2=0x55703e5eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 -- 172.21.15.37:0/2276845641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 msgr2=0x55703e5f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.418+0000 7f5b927fc700 1 -- 172.21.15.37:0/2276845641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703e484880 con 0x55703e5e4d50 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5b927fc700 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809c630 tx=0x7f5b8809c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d19a0cb86d9815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5b79ffb700 1 -- 172.21.15.37:0/2276845641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8809c6a0 con 0x55703e5e4d50 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5b79ffb700 1 -- 172.21.15.37:0/2276845641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b880a0a50 con 0x55703e5e4d50 2024-04-26T18:23:33.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5b79ffb700 1 -- 172.21.15.37:0/2276845641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8809cb00 con 0x55703e5e4d50 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/2276845641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 msgr2=0x55703e5e51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.419+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809c630 tx=0x7f5b8809c230 comp rx=0 tx=0).stop 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/2276845641 shutdown_connections 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/2276845641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5e51b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/2276845641 >> 172.21.15.37:0/2276845641 conn(0x55703e513570 msgr2=0x55703e5df900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/2276845641 shutdown_connections 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/2276845641 wait complete. 2024-04-26T18:23:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- start start 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 Processor -- start 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703e5f1cb0 con 0x55703e5eaa20 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55703e4f1a30 con 0x55703e5e56f0 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55703e5f1e30 con 0x55703e5e4d50 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.420+0000 7f5ba42d0b80 10 client.0 _ll_register_callbacks cb 0x55703e4cf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b92ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b92ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:40810/0 (socket says 172.21.15.37:40810) 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36866/0 (socket says 172.21.15.37:36866) 2024-04-26T18:23:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b92ffd700 1 -- 172.21.15.37:0/1861706631 learned_addr learned my addr 172.21.15.37:0/1861706631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b927fc700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 -- 172.21.15.37:0/1861706631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 msgr2=0x55703e5f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 -- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 msgr2=0x55703e5ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 -- 172.21.15.37:0/1861706631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703e4f5430 con 0x55703e5eaa20 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b937fe700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009fd0 tx=0x7f5b8c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d481ef7beb1e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0076e0 con 0x55703e5eaa20 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f5b8c007880 con 0x55703e5eaa20 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.421+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b880a0a50 con 0x55703e5eaa20 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c003e20 con 0x55703e5eaa20 2024-04-26T18:23:33.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 10 client.14982 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55703e5fc470 con 0x55703e5eaa20 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b880a0a50 con 0x55703e5eaa20 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5b8c02f070 con 0x55703e5eaa20 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 10 client.14982 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 20 client.14982 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e5fea50 0x55703e5ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 4 client.14982 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55703e5451c0 con 0x55703e5fea50 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.422+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e5fea50 0x55703e5ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.423+0000 7f5b72ffd700 10 client.14982 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.423+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e5fea50 0x55703e5ffec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95d926d16fd798c server_cookie=5c7f56c8e6395694 in_seq=0 out_seq=0 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.423+0000 7f5b72ffd700 1 -- 172.21.15.37:0/1861706631 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55703e5451c0 con 0x55703e5fea50 2024-04-26T18:23:33.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.423+0000 7f5b72ffd700 10 client.14982 handle_command_reply: tid=0 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 client.14982 shutdown 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 msgr2=0x55703e5eec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009fd0 tx=0x7f5b8c003c30 comp rx=0 tx=0).stop 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 shutdown_connections 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e5fea50 0x55703e5ffec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5eaa20 0x55703e5eec20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1861706631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 >> 172.21.15.37:0/1861706631 conn(0x55703e513570 msgr2=0x55703e5e3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5b72ffd700 0 client.14982 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 shutdown_connections 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1861706631 wait complete. 2024-04-26T18:23:33.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.424+0000 7f5ba42d0b80 20 client.14982 trim_cache size 0 max 16384 2024-04-26T18:23:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.444+0000 7f5ba42d0b80 1 Processor -- start 2024-04-26T18:23:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.444+0000 7f5ba42d0b80 1 -- start start 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703e5fe200 con 0x55703e5e4d50 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55703e5fe8d0 con 0x55703e5e56f0 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55703e5fefa0 con 0x55703e5eaa20 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36874/0 (socket says 172.21.15.37:36874) 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 -- 172.21.15.37:0/1456591264 learned_addr learned my addr 172.21.15.37:0/1456591264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b927fc700 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 -- 172.21.15.37:0/1456591264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 msgr2=0x55703e5e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 -- 172.21.15.37:0/1456591264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 msgr2=0x55703e5e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 -- 172.21.15.37:0/1456591264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55703e5fb0d0 con 0x55703e5e4d50 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b84007fe0 tx=0x7f5b84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9292bda57578e43b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b797fa700 1 -- 172.21.15.37:0/1456591264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8402c030 con 0x55703e5e4d50 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b797fa700 1 -- 172.21.15.37:0/1456591264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b84024cc0 con 0x55703e5e4d50 2024-04-26T18:23:33.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.445+0000 7f5b797fa700 1 -- 172.21.15.37:0/1456591264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84027690 con 0x55703e5e4d50 2024-04-26T18:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1456591264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 msgr2=0x55703e5f1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b84007fe0 tx=0x7f5b84013000 comp rx=0 tx=0).stop 2024-04-26T18:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1456591264 shutdown_connections 2024-04-26T18:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5eaa20 0x55703e5e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e4d50 0x55703e5f1940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/1456591264 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5e56f0 0x55703e5e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1456591264 >> 172.21.15.37:0/1456591264 conn(0x55703e513570 msgr2=0x55703e5ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1456591264 shutdown_connections 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.446+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/1456591264 wait complete. 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 -- start start 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 Processor -- start 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 0x55703e5e7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55703e5ffd40 con 0x55703e5e56f0 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55703e5ff670 con 0x55703e5eaa20 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55703e600410 con 0x55703e5e4d50 2024-04-26T18:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5ba42d0b80 10 client.0 _ll_register_callbacks cb 0x55703e4cf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36890/0 (socket says 172.21.15.37:36890) 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 -- 172.21.15.37:0/3785404200 learned_addr learned my addr 172.21.15.37:0/3785404200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 -- 172.21.15.37:0/3785404200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 msgr2=0x55703e5e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b937fe700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 0x55703e5e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 -- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 msgr2=0x55703e5e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 0x55703e5e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b927fc700 1 -- 172.21.15.37:0/3785404200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703e605450 con 0x55703e5e56f0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.447+0000 7f5b937fe700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 0x55703e5e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b927fc700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809c3a0 tx=0x7f5b880a0b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4aea0a376b77ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88065070 con 0x55703e5e56f0 2024-04-26T18:23:33.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f5b8800f740 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b84024cc0 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b88082050 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 10 client.14988 fetch_fsmap learned FSMap version 25 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55703e665e00 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b84024cc0 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5b880a4e80 con 0x55703e5e56f0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 10 client.14988 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 20 client.14988 populate_metadata read hostname 'smithi037' 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e66a4c0 0x55703e6691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 4 client.14988 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.448+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "status"}) -- 0x55703e53f260 con 0x55703e66a4c0 2024-04-26T18:23:33.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e66a4c0 0x55703e6691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5b92ffd700 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e66a4c0 0x55703e6691d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85fb231c3dce95c server_cookie=f9d7c385a16fe926 in_seq=0 out_seq=0 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5b72ffd700 10 client.14988 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5b72ffd700 1 -- 172.21.15.37:0/3785404200 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55703e53f260 con 0x55703e66a4c0 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5b72ffd700 10 client.14988 handle_command_reply: tid=0 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5ba42d0b80 1 client.14988 shutdown 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 msgr2=0x55703e5e71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.449+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b8809c3a0 tx=0x7f5b880a0b90 comp rx=0 tx=0).stop 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 shutdown_connections 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55703e66a4c0 0x55703e6691d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55703e5eaa20 0x55703e5e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55703e5e4d50 0x55703e5e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 --2- 172.21.15.37:0/3785404200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55703e5e56f0 0x55703e5e71c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 >> 172.21.15.37:0/3785404200 conn(0x55703e513570 msgr2=0x55703e5df4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b72ffd700 0 client.14988 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 shutdown_connections 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 1 -- 172.21.15.37:0/3785404200 wait complete. 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5ba42d0b80 20 client.14988 trim_cache size 0 max 16384 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24464, 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:23:33.455 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 141.54473369999999, 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 157.91666595199999 2024-04-26T18:23:33.456 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 msgr2=0x7f5b7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 0x7f5b7c0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840099d0 tx=0x7f5b84008040 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 msgr2=0x7f5b941028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c9b0 tx=0x7f5b8800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 shutdown_connections 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f5b7c0a3c40 0x7f5b7c0a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b941094d0 0x7f5b94102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b9406bbd0 0x7f5b94102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 --2- 172.21.15.37:0/2693586463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b9406c590 0x7f5b941028d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 >> 172.21.15.37:0/2693586463 conn(0x7f5b940793c0 msgr2=0x7f5b941001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.450+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 shutdown_connections 2024-04-26T18:23:33.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:33.451+0000 7f5b99a3e700 1 -- 172.21.15.37:0/2693586463 wait complete. 2024-04-26T18:23:34.150 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T18:23:34.150 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:23:34.160 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:23:34.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T18:23:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:34.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:34 smithi037 ceph-mon[29171]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:34.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:34 smithi120 ceph-mon[32244]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:34 smithi170 ceph-mon[32697]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- 172.21.15.37:0/1690112417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1035a0 msgr2=0x7f520c103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 --2- 172.21.15.37:0/1690112417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1035a0 0x7f520c103a00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0099c0 tx=0x7f51fc009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- 172.21.15.37:0/1690112417 shutdown_connections 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 --2- 172.21.15.37:0/1690112417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c103f40 0x7f520c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 --2- 172.21.15.37:0/1690112417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c1023a0 0x7f520c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 --2- 172.21.15.37:0/1690112417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1035a0 0x7f520c103a00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- 172.21.15.37:0/1690112417 >> 172.21.15.37:0/1690112417 conn(0x7f520c0fd940 msgr2=0x7f520c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- 172.21.15.37:0/1690112417 shutdown_connections 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- 172.21.15.37:0/1690112417 wait complete. 2024-04-26T18:23:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 Processor -- start 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.971+0000 7f521415c700 1 -- start start 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 0x7f520c104980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c103f40 0x7f520c104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10de10 con 0x7f520c1023a0 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10dc90 con 0x7f520c103f40 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10df90 con 0x7f520c1035a0 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33506/0 (socket says 172.21.15.37:33506) 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 -- 172.21.15.37:0/3993201649 learned_addr learned my addr 172.21.15.37:0/3993201649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f52126f9700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c103f40 0x7f520c104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f52116f7700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 0x7f520c104980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 -- 172.21.15.37:0/3993201649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 msgr2=0x7f520c104980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 0x7f520c104980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 -- 172.21.15.37:0/3993201649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c103f40 msgr2=0x7f520c104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c103f40 0x7f520c104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 -- 172.21.15.37:0/3993201649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c105790 con 0x7f520c1023a0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f52116f7700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 0x7f520c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5211ef8700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f520800c970 tx=0x7f520800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208004220 con 0x7f520c1023a0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c1a36f0 con 0x7f520c1023a0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208007d50 con 0x7f520c1023a0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208003d00 con 0x7f520c1023a0 2024-04-26T18:23:35.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.972+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a3c50 con 0x7f520c1023a0 2024-04-26T18:23:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.973+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c06a110 con 0x7f520c1023a0 2024-04-26T18:23:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.975+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52080078a0 con 0x7f520c1023a0 2024-04-26T18:23:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.976+0000 7f5202ffd700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 0x7f51f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.976+0000 7f52116f7700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 0x7f51f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.976+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f52080d0440 con 0x7f520c1023a0 2024-04-26T18:23:35.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.976+0000 7f52116f7700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 0x7f51f80a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51fc003890 tx=0x7f51fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:35.979+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5208014030 con 0x7f520c1023a0 2024-04-26T18:23:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.208+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f520c0ffb30 con 0x7f51f80a3c40 2024-04-26T18:23:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.209+0000 7f5202ffd700 1 -- 172.21.15.37:0/3993201649 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f520c0ffb30 con 0x7f51f80a3c40 2024-04-26T18:23:36.209 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-04-26T18:23:36.209 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-04-26T18:23:36.209 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 msgr2=0x7f51f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 0x7f51f80a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f51fc003890 tx=0x7f51fc005fb0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 msgr2=0x7f520c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f520800c970 tx=0x7f520800cc80 comp rx=0 tx=0).stop 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 shutdown_connections 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f51f80a3c40 0x7f51f80a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f520c103f40 0x7f520c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f520c1023a0 0x7f520c1062e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 --2- 172.21.15.37:0/3993201649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f520c1035a0 0x7f520c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 >> 172.21.15.37:0/3993201649 conn(0x7f520c0fd940 msgr2=0x7f520c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 shutdown_connections 2024-04-26T18:23:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:36.211+0000 7f521415c700 1 -- 172.21.15.37:0/3993201649 wait complete. 2024-04-26T18:23:36.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:36 smithi037 ceph-mon[29171]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:36.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:36 smithi120 ceph-mon[32244]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:36.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:36 smithi170 ceph-mon[32697]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:36.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T18:23:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:37.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:37 smithi037 ceph-mon[29171]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:37 smithi120 ceph-mon[32244]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:37 smithi170 ceph-mon[32697]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:38.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:38 smithi037 ceph-mon[29171]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:38.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:38 smithi170 ceph-mon[32697]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:38.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:38 smithi120 ceph-mon[32244]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3082267174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1035c0 msgr2=0x7f8c7c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3082267174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c103a20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8c780099c0 tx=0x7f8c78009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3082267174 shutdown_connections 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3082267174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c103f60 0x7f8c7c10a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3082267174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3082267174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c103a20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.764+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3082267174 >> 172.21.15.37:0/3082267174 conn(0x7f8c7c0fd980 msgr2=0x7f8c7c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3082267174 shutdown_connections 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3082267174 wait complete. 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 Processor -- start 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- start start 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c1049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 0x7f8c7c104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10dfa0 con 0x7f8c7c1023c0 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10de20 con 0x7f8c7c103f60 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.765+0000 7f8c83ca6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c10e120 con 0x7f8c7c1035c0 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33536/0 (socket says 172.21.15.37:33536) 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 -- 172.21.15.37:0/3891685524 learned_addr learned my addr 172.21.15.37:0/3891685524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81241700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c1049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c82243700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 0x7f8c7c104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 -- 172.21.15.37:0/3891685524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c1035c0 msgr2=0x7f8c7c1049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c1049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 -- 172.21.15.37:0/3891685524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 msgr2=0x7f8c7c104ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 0x7f8c7c104ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 -- 172.21.15.37:0/3891685524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c105720 con 0x7f8c7c1023c0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c82243700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 0x7f8c7c104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c81a42700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b500 tx=0x7f8c6c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c015020 con 0x7f8c7c1023c0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c1a3880 con 0x7f8c7c1023c0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c6c011ce0 con 0x7f8c7c1023c0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.766+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c010d20 con 0x7f8c7c1023c0 2024-04-26T18:23:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.767+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c1a3e90 con 0x7f8c7c1023c0 2024-04-26T18:23:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.767+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c06a110 con 0x7f8c7c1023c0 2024-04-26T18:23:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.769+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8c6c018020 con 0x7f8c7c1023c0 2024-04-26T18:23:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.769+0000 7f8c72ffd700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 0x7f8c680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.769+0000 7f8c81241700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 0x7f8c680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.769+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f8c6c0d00f0 con 0x7f8c7c1023c0 2024-04-26T18:23:38.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.770+0000 7f8c81241700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 0x7f8c680a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c78006010 tx=0x7f8c78003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.773+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c6c01f020 con 0x7f8c7c1023c0 2024-04-26T18:23:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:38.988+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8c7c0ffb40 con 0x7f8c680a3c40 2024-04-26T18:23:39.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.002+0000 7f8c72ffd700 1 -- 172.21.15.37:0/3891685524 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f8c7c0ffb40 con 0x7f8c680a3c40 2024-04-26T18:23:39.002 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (2m) 25s ago 2m 15.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b f30d4c996070 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi120 running (2m) 27s ago 2m 27.9M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b c1cdb629633f 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi170 running (2m) 27s ago 2m 20.5M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b bc916e20a0de 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (3m) 25s ago 3m 14.7M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b a4f4b1ef543c 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi120 running (3m) 27s ago 3m 19.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b cf4c59280a5f 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi170 running (3m) 27s ago 3m 20.8M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b b5ad78d08d38 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (2m) 25s ago 2m 17.0M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 0dc2a5a0550d 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi120 running (2m) 27s ago 2m 20.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b f68f429048b7 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi170 running (2m) 27s ago 2m 22.0M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 1bdd765c1791 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (3m) 25s ago 3m 14.6M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 6a25cf689be6 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi120 running (3m) 27s ago 3m 16.2M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b f7106543ce3d 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi170 running (3m) 27s ago 3m 18.7M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b ea0891bac226 2024-04-26T18:23:39.003 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (10m) 25s ago 10m 516M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 8b820eaa5451 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi120 *:8443,8765 running (8m) 27s ago 8m 453M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b a21af0e1dac6 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi170 *:8443,8765 running (8m) 27s ago 8m 454M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 694f87fa27e9 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (10m) 25s ago 10m 51.5M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 631bb5bf4720 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi120 running (8m) 27s ago 8m 40.8M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 2cbb34997323 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi170 running (8m) 27s ago 8m 41.0M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b c1339d7ac19a 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 25s ago 8m 60.2M 4273M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 07bf2db1e663 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi120 running (7m) 27s ago 7m 51.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 794c01f9683b 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi170 running (7m) 27s ago 7m 58.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b db70db4000a2 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (7m) 25s ago 7m 47.2M 4273M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 10aad971f99e 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi120 running (6m) 27s ago 6m 54.5M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 4cc61f65380a 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi170 running (6m) 27s ago 6m 50.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b a76da6ab02fa 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (5m) 25s ago 5m 53.7M 4273M 19.0.0-3267-gbc94a2a9 3a6bde214c5b fb64ddd809bd 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi120 running (5m) 27s ago 5m 50.9M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 6af3b49f75dc 2024-04-26T18:23:39.004 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi170 running (4m) 27s ago 4m 47.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 09a3f0d85d2d 2024-04-26T18:23:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (4m) 25s ago 4m 47.7M 4273M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 3294eec250f9 2024-04-26T18:23:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi120 running (4m) 27s ago 4m 50.6M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b e67063732d9f 2024-04-26T18:23:39.005 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi170 running (3m) 27s ago 3m 52.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b e2fdb0f83876 2024-04-26T18:23:39.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 msgr2=0x7f8c680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 0x7f8c680a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8c78006010 tx=0x7f8c78003860 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 msgr2=0x7f8c7c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00b500 tx=0x7f8c6c00b810 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 shutdown_connections 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f8c680a3c40 0x7f8c680a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c7c103f60 0x7f8c7c104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c7c1023c0 0x7f8c7c1062e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 --2- 172.21.15.37:0/3891685524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c7c1035c0 0x7f8c7c1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.004+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 >> 172.21.15.37:0/3891685524 conn(0x7f8c7c0fd980 msgr2=0x7f8c7c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.005+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 shutdown_connections 2024-04-26T18:23:39.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:39.005+0000 7f8c83ca6700 1 -- 172.21.15.37:0/3891685524 wait complete. 2024-04-26T18:23:39.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T18:23:39.907 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:40.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:40 smithi037 ceph-mon[29171]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:40.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:40 smithi037 ceph-mon[29171]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:40.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:40 smithi120 ceph-mon[32244]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:40.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:40 smithi120 ceph-mon[32244]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:40.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:40 smithi170 ceph-mon[32697]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:40.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:40 smithi170 ceph-mon[32697]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:41.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 -- 172.21.15.37:0/2827853765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6281068e0 msgr2=0x7ff628108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 --2- 172.21.15.37:0/2827853765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6281068e0 0x7ff628108cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009990 tx=0x7ff61c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 -- 172.21.15.37:0/2827853765 shutdown_connections 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 --2- 172.21.15.37:0/2827853765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff628109210 0x7ff62810b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 --2- 172.21.15.37:0/2827853765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 0x7ff6281063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 --2- 172.21.15.37:0/2827853765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff6281068e0 0x7ff628108cd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.513+0000 7ff6300e3700 1 -- 172.21.15.37:0/2827853765 >> 172.21.15.37:0/2827853765 conn(0x7ff6280fd940 msgr2=0x7ff6280ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- 172.21.15.37:0/2827853765 shutdown_connections 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- 172.21.15.37:0/2827853765 wait complete. 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 Processor -- start 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- start start 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 0x7ff62819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 0x7ff62819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:41.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810ef80 con 0x7ff628109210 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810ee00 con 0x7ff6281068e0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff6300e3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810f100 con 0x7ff628103fb0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff62e680700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff62e680700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33550/0 (socket says 172.21.15.37:33550) 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.514+0000 7ff62e680700 1 -- 172.21.15.37:0/3532155909 learned_addr learned my addr 172.21.15.37:0/3532155909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62de7f700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 0x7ff62819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62d67e700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 0x7ff62819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 -- 172.21.15.37:0/3532155909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 msgr2=0x7ff62819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 0x7ff62819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 -- 172.21.15.37:0/3532155909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 msgr2=0x7ff62819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 0x7ff62819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 -- 172.21.15.37:0/3532155909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a3420 con 0x7ff628109210 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62d67e700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 0x7ff62819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff62e680700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff62000b4b0 tx=0x7ff62000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:41.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620015020 con 0x7ff628109210 2024-04-26T18:23:41.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff620011ce0 con 0x7ff628109210 2024-04-26T18:23:41.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a3710 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620010d40 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.515+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a3cd0 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.516+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62806a110 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.518+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff620018020 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.518+0000 7ff626ffd700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 0x7ff6100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.518+0000 7ff62de7f700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 0x7ff6100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.518+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7ff6200cfb80 con 0x7ff628109210 2024-04-26T18:23:41.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.519+0000 7ff62de7f700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 0x7ff6100a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff61800e270 tx=0x7ff618009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:41.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.523+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff62001f020 con 0x7ff628109210 2024-04-26T18:23:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.738+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff628063860 con 0x7ff6100a3cc0 2024-04-26T18:23:41.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.744+0000 7ff626ffd700 1 -- 172.21.15.37:0/3532155909 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff628063860 con 0x7ff6100a3cc0 2024-04-26T18:23:41.744 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T18:23:41.744 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 29s ago 3m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-04-26T18:23:41.744 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 29s ago 8m smithi037=x;smithi120=y;smithi170=z;count:3 2024-04-26T18:23:41.744 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 29s ago 9m smithi037:172.21.15.37=a;smithi120:172.21.15.120=b;smithi170:172.21.15.170=c;count:3 2024-04-26T18:23:41.744 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 29s ago - 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 msgr2=0x7ff6100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 0x7ff6100a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff61800e270 tx=0x7ff618009040 comp rx=0 tx=0).stop 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 msgr2=0x7ff6281a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff62000b4b0 tx=0x7ff62000b870 comp rx=0 tx=0).stop 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 shutdown_connections 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7ff6100a3cc0 0x7ff6100a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628109210 0x7ff6281a2d80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff628103fb0 0x7ff62819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 --2- 172.21.15.37:0/3532155909 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6281068e0 0x7ff62819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 >> 172.21.15.37:0/3532155909 conn(0x7ff6280fd940 msgr2=0x7ff628103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 shutdown_connections 2024-04-26T18:23:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:41.746+0000 7ff6300e3700 1 -- 172.21.15.37:0/3532155909 wait complete. 2024-04-26T18:23:42.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:42 smithi037 ceph-mon[29171]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:42.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:42 smithi120 ceph-mon[32244]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:42.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T18:23:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:42 smithi170 ceph-mon[32697]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:43.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:43 smithi037 ceph-mon[29171]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:43 smithi120 ceph-mon[32244]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:43 smithi170 ceph-mon[32697]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:44.149 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:44 smithi037 ceph-mon[29171]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 -- 172.21.15.37:0/168534531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0102380 msgr2=0x7f01b01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 --2- 172.21.15.37:0/168534531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0102380 0x7f01b01027a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f01a40099c0 tx=0x7f01a4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 -- 172.21.15.37:0/168534531 shutdown_connections 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 --2- 172.21.15.37:0/168534531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103f20 0x7f01b010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 --2- 172.21.15.37:0/168534531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0102380 0x7f01b01027a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 --2- 172.21.15.37:0/168534531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0103580 0x7f01b01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.383+0000 7f01b788e700 1 -- 172.21.15.37:0/168534531 >> 172.21.15.37:0/168534531 conn(0x7f01b00fd920 msgr2=0x7f01b00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- 172.21.15.37:0/168534531 shutdown_connections 2024-04-26T18:23:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- 172.21.15.37:0/168534531 wait complete. 2024-04-26T18:23:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 Processor -- start 2024-04-26T18:23:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- start start 2024-04-26T18:23:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0102380 0x7f01b0106310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:44.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103580 0x7f01b0104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010dec0 con 0x7f01b0103f20 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010dd40 con 0x7f01b0102380 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b788e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010e040 con 0x7f01b0103580 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.384+0000 7f01b5e2b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33562/0 (socket says 172.21.15.37:33562) 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 -- 172.21.15.37:0/1711106214 learned_addr learned my addr 172.21.15.37:0/1711106214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b562a700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0102380 0x7f01b0106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b4e29700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103580 0x7f01b0104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 -- 172.21.15.37:0/1711106214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103580 msgr2=0x7f01b0104960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103580 0x7f01b0104960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 -- 172.21.15.37:0/1711106214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0102380 msgr2=0x7f01b0106310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0102380 0x7f01b0106310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 -- 172.21.15.37:0/1711106214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b0105750 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b5e2b700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f01a800b970 tx=0x7f01a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800d860 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a800e430 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b01a3800 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017660 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.385+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b01a3d40 con 0x7f01b0103f20 2024-04-26T18:23:44.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.386+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b006a110 con 0x7f01b0103f20 2024-04-26T18:23:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.387+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f01a800da00 con 0x7f01b0103f20 2024-04-26T18:23:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.388+0000 7f01a27fc700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 0x7f019c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.388+0000 7f01b562a700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 0x7f019c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.388+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f01a80d02f0 con 0x7f01b0103f20 2024-04-26T18:23:44.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.389+0000 7f01b562a700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 0x7f019c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009f50 tx=0x7f01a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.392+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01a8013030 con 0x7f01b0103f20 2024-04-26T18:23:44.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:44 smithi120 ceph-mon[32244]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:44 smithi170 ceph-mon[32697]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.606+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f01b00ffb50 con 0x7f019c0a3cc0 2024-04-26T18:23:44.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.607+0000 7f01a27fc700 1 -- 172.21.15.37:0/1711106214 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f01b00ffb50 con 0x7f019c0a3cc0 2024-04-26T18:23:44.607 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-04-26T18:23:44.607 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-04-26T18:23:44.608 INFO:teuthology.orchestra.run.smithi037.stdout:smithi120 172.21.15.120 2024-04-26T18:23:44.608 INFO:teuthology.orchestra.run.smithi037.stdout:smithi170 172.21.15.170 2024-04-26T18:23:44.608 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.609+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 msgr2=0x7f019c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 0x7f019c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009f50 tx=0x7f01a400b540 comp rx=0 tx=0).stop 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 msgr2=0x7f01b0104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f01a800b970 tx=0x7f01a800bc80 comp rx=0 tx=0).stop 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 shutdown_connections 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f019c0a3cc0 0x7f019c0a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01b0103f20 0x7f01b0104ea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01b0102380 0x7f01b0106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 --2- 172.21.15.37:0/1711106214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01b0103580 0x7f01b0104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 >> 172.21.15.37:0/1711106214 conn(0x7f01b00fd920 msgr2=0x7f01b00ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 shutdown_connections 2024-04-26T18:23:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:44.610+0000 7f01b788e700 1 -- 172.21.15.37:0/1711106214 wait complete. 2024-04-26T18:23:45.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T18:23:45.523 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:46.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:46 smithi037 ceph-mon[29171]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:46.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:46 smithi037 ceph-mon[29171]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:46.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:46 smithi120 ceph-mon[32244]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:46 smithi120 ceph-mon[32244]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:46 smithi170 ceph-mon[32697]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:46 smithi170 ceph-mon[32697]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 -- 172.21.15.37:0/718734804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d41068e0 msgr2=0x7f43d4108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 --2- 172.21.15.37:0/718734804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d41068e0 0x7f43d4108cd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 -- 172.21.15.37:0/718734804 shutdown_connections 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 --2- 172.21.15.37:0/718734804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4109210 0x7f43d410b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 --2- 172.21.15.37:0/718734804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4103fb0 0x7f43d41063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 --2- 172.21.15.37:0/718734804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d41068e0 0x7f43d4108cd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 -- 172.21.15.37:0/718734804 >> 172.21.15.37:0/718734804 conn(0x7f43d40fd920 msgr2=0x7f43d40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:47.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 -- 172.21.15.37:0/718734804 shutdown_connections 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.214+0000 7f43d8953700 1 -- 172.21.15.37:0/718734804 wait complete. 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 Processor -- start 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 -- start start 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 0x7f43d419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4109210 0x7f43d41a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410ef00 con 0x7f43d4103fb0 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410ed80 con 0x7f43d4109210 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d8953700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43d410f080 con 0x7f43d41068e0 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60568/0 (socket says 172.21.15.37:60568) 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 -- 172.21.15.37:0/3469414721 learned_addr learned my addr 172.21.15.37:0/3469414721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d2d9d700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4109210 0x7f43d41a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:47.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d1d9b700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 0x7f43d419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 -- 172.21.15.37:0/3469414721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 msgr2=0x7f43d419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 0x7f43d419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 -- 172.21.15.37:0/3469414721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4109210 msgr2=0x7f43d41a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4109210 0x7f43d41a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d259c700 1 -- 172.21.15.37:0/3469414721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d41a3490 con 0x7f43d4103fb0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.215+0000 7f43d1d9b700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 0x7f43d419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43d259c700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f43c400add0 tx=0x7f43c400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4017020 con 0x7f43d4103fb0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d41a3780 con 0x7f43d4103fb0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c400cdf0 con 0x7f43d4103fb0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c40053e0 con 0x7f43d4103fb0 2024-04-26T18:23:47.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.216+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41a3d10 con 0x7f43d4103fb0 2024-04-26T18:23:47.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.217+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d406a110 con 0x7f43d4103fb0 2024-04-26T18:23:47.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.219+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f43c4004ad0 con 0x7f43d4103fb0 2024-04-26T18:23:47.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.219+0000 7f43cb7fe700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 0x7f43c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:47.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.219+0000 7f43d1d9b700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 0x7f43c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:47.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.219+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f43c40d0190 con 0x7f43d4103fb0 2024-04-26T18:23:47.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.220+0000 7f43d1d9b700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 0x7f43c00a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5c0 tx=0x7f43bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:47.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.223+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43c4021020 con 0x7f43d4103fb0 2024-04-26T18:23:47.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.441+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f43d4063860 con 0x7f43c00a3c40 2024-04-26T18:23:47.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.442+0000 7f43cb7fe700 1 -- 172.21.15.37:0/3469414721 <== mgr.14152 v2:172.21.15.37:6800/813673186 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f43d4063860 con 0x7f43c00a3c40 2024-04-26T18:23:47.443 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T18:23:47.443 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:23:47.443 INFO:teuthology.orchestra.run.smithi037.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:23:47.443 INFO:teuthology.orchestra.run.smithi037.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-26T18:23:47.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 msgr2=0x7f43c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:47.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 0x7f43c00a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00b5c0 tx=0x7f43bc005fd0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 msgr2=0x7f43d419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:47.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f43c400add0 tx=0x7f43c400c330 comp rx=0 tx=0).stop 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 shutdown_connections 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f43c00a3c40 0x7f43c00a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4109210 0x7f43d41a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4103fb0 0x7f43d419b830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 --2- 172.21.15.37:0/3469414721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d41068e0 0x7f43d419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 >> 172.21.15.37:0/3469414721 conn(0x7f43d40fd920 msgr2=0x7f43d4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 shutdown_connections 2024-04-26T18:23:47.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:47.445+0000 7f43d8953700 1 -- 172.21.15.37:0/3469414721 wait complete. 2024-04-26T18:23:48.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:48 smithi037 ceph-mon[29171]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:48.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:48 smithi037 ceph-mon[29171]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:48.365 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:23:48.374 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:23:48.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T18:23:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:48 smithi120 ceph-mon[32244]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:48.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:48 smithi120 ceph-mon[32244]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:48 smithi170 ceph-mon[32697]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:48.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:48 smithi170 ceph-mon[32697]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:48.630 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:50.350 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:50 smithi037 ceph-mon[29171]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:50.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 -- 172.21.15.37:0/204575335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec106840 msgr2=0x7f37ec108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 --2- 172.21.15.37:0/204575335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec106840 0x7f37ec108c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009990 tx=0x7f37dc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 -- 172.21.15.37:0/204575335 shutdown_connections 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 --2- 172.21.15.37:0/204575335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec109170 0x7f37ec10b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 --2- 172.21.15.37:0/204575335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec103f10 0x7f37ec106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 --2- 172.21.15.37:0/204575335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec106840 0x7f37ec108c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 -- 172.21.15.37:0/204575335 >> 172.21.15.37:0/204575335 conn(0x7f37ec0fd920 msgr2=0x7f37ec0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 -- 172.21.15.37:0/204575335 shutdown_connections 2024-04-26T18:23:50.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.373+0000 7f37f3b51700 1 -- 172.21.15.37:0/204575335 wait complete. 2024-04-26T18:23:50.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 Processor -- start 2024-04-26T18:23:50.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 -- start start 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec106840 0x7f37ec199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec109170 0x7f37ec1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10ef40 con 0x7f37ec103f10 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10edc0 con 0x7f37ec109170 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f3b51700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10f0c0 con 0x7f37ec106840 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.374+0000 7f37f18ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60586/0 (socket says 172.21.15.37:60586) 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 -- 172.21.15.37:0/868917452 learned_addr learned my addr 172.21.15.37:0/868917452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f20ee700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec109170 0x7f37ec1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f10ec700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec106840 0x7f37ec199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:50.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 -- 172.21.15.37:0/868917452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec106840 msgr2=0x7f37ec199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec106840 0x7f37ec199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 -- 172.21.15.37:0/868917452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec109170 msgr2=0x7f37ec1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec109170 0x7f37ec1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 -- 172.21.15.37:0/868917452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1a1280 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f18ed700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37e800ef40 tx=0x7f37e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e800ca70 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e8019070 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a1510 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80146e0 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.375+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a1aa0 con 0x7f37ec103f10 2024-04-26T18:23:50.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.376+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec06a110 con 0x7f37ec103f10 2024-04-26T18:23:50.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.377+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f37e8012050 con 0x7f37ec103f10 2024-04-26T18:23:50.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.378+0000 7f37e2ffd700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 0x7f37d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:50.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.378+0000 7f37f10ec700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 0x7f37d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:50.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.378+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f37e80d0a80 con 0x7f37ec103f10 2024-04-26T18:23:50.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.378+0000 7f37f10ec700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 0x7f37d80a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b2e0 tx=0x7f37dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:50.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.382+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37e8010040 con 0x7f37ec103f10 2024-04-26T18:23:50.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:50 smithi120 ceph-mon[32244]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:50.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:50 smithi170 ceph-mon[32697]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:50.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.635+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f37ec064030 con 0x7f37ec103f10 2024-04-26T18:23:50.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.636+0000 7f37e2ffd700 1 -- 172.21.15.37:0/868917452 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3058 (secure 0 0 0) 0x7f37e809d430 con 0x7f37ec103f10 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:e25 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 25 2024-04-26T18:23:50.637 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-26T18:21:10.996859+0000 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-26T18:21:29.674118+0000 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-04-26T18:23:50.638 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 3 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24464,1=24449,2=24442} 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-04-26T18:23:50.639 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24464 members: 24464,24442,24449 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24464} state up:active seq 6 addr [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{0:24430} state up:standby-replay seq 1 addr [v2:172.21.15.170:6836/2099915678,v1:172.21.15.170:6837/2099915678] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{1:24449} state up:active seq 12 addr [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{1:24425} state up:standby-replay seq 1 addr [v2:172.21.15.120:6834/3605086601,v1:172.21.15.120:6835/3605086601] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{2:24442} state up:active seq 12 addr [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{2:24424} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/707687219,v1:172.21.15.170:6835/707687219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:50.640 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.37:6834/1294150322,v1:172.21.15.37:6835/1294150322] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.641 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.37:6836/1073340191,v1:172.21.15.37:6837/1073340191] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.641 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14583} state up:standby seq 1 addr [v2:172.21.15.37:6838/1645821272,v1:172.21.15.37:6839/1645821272] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.641 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14595} state up:standby seq 1 addr [v2:172.21.15.37:6840/3026662936,v1:172.21.15.37:6841/3026662936] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.641 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.170:6832/1924785212,v1:172.21.15.170:6833/1924785212] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.641 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24401} state up:standby seq 1 addr [v2:172.21.15.120:6832/1778061281,v1:172.21.15.120:6833/1778061281] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 msgr2=0x7f37d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 0x7f37d80a6180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00b2e0 tx=0x7f37dc01a040 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 msgr2=0x7f37ec199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37e800ef40 tx=0x7f37e800c3a0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 shutdown_connections 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f37d80a3cc0 0x7f37d80a6180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37ec109170 0x7f37ec1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37ec103f10 0x7f37ec199620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 --2- 172.21.15.37:0/868917452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37ec106840 0x7f37ec199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 >> 172.21.15.37:0/868917452 conn(0x7f37ec0fd920 msgr2=0x7f37ec105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 shutdown_connections 2024-04-26T18:23:50.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:50.638+0000 7f37f3b51700 1 -- 172.21.15.37:0/868917452 wait complete. 2024-04-26T18:23:50.643 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:23:51.329 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:51 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/868917452' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:23:51.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T18:23:51.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:51 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/868917452' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:23:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:51 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/868917452' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:23:51.608 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:52.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:52 smithi037 ceph-mon[29171]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:52.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:52 smithi120 ceph-mon[32244]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:52.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:52 smithi170 ceph-mon[32697]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2490431445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc104680 msgr2=0x7f3abc104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2490431445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc104680 0x7f3abc104ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0099c0 tx=0x7f3aac009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2490431445 shutdown_connections 2024-04-26T18:23:53.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2490431445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc105020 0x7f3abc10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2490431445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc103480 0x7f3abc1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2490431445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc104680 0x7f3abc104ae0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2490431445 >> 172.21.15.37:0/2490431445 conn(0x7f3abc0fea20 msgr2=0x7f3abc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2490431445 shutdown_connections 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2490431445 wait complete. 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.309+0000 7f3ac3a1a700 1 Processor -- start 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 -- start start 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc104680 0x7f3abc075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc105020 0x7f3abc075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10ef70 con 0x7f3abc103480 2024-04-26T18:23:53.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10edf0 con 0x7f3abc105020 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac3a1a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10f0f0 con 0x7f3abc104680 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60598/0 (socket says 172.21.15.37:60598) 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 -- 172.21.15.37:0/2297350822 learned_addr learned my addr 172.21.15.37:0/2297350822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac1fb7700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc105020 0x7f3abc075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac0fb5700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc104680 0x7f3abc075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 -- 172.21.15.37:0/2297350822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc104680 msgr2=0x7f3abc075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc104680 0x7f3abc075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 -- 172.21.15.37:0/2297350822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc105020 msgr2=0x7f3abc075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc105020 0x7f3abc075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 -- 172.21.15.37:0/2297350822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc076600 con 0x7f3abc103480 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ac17b6700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800d980 tx=0x7f3ab800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:53.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.310+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8017030 con 0x7f3abc103480 2024-04-26T18:23:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.311+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab80044d0 con 0x7f3abc103480 2024-04-26T18:23:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.311+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8003970 con 0x7f3abc103480 2024-04-26T18:23:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.311+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc1a3780 con 0x7f3abc103480 2024-04-26T18:23:53.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.311+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc1a3c40 con 0x7f3abc103480 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.312+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc06a110 con 0x7f3abc103480 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.313+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ab801a070 con 0x7f3abc103480 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.313+0000 7f3ab27fc700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 0x7f3aa80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.314+0000 7f3ac0fb5700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 0x7f3aa80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.314+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f3ab80cfa50 con 0x7f3abc103480 2024-04-26T18:23:53.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.314+0000 7f3ac0fb5700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 0x7f3aa80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009990 tx=0x7f3aac00b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:53.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.317+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ab809c400 con 0x7f3abc103480 2024-04-26T18:23:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.527+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3abc064030 con 0x7f3abc103480 2024-04-26T18:23:53.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.528+0000 7f3ab27fc700 1 -- 172.21.15.37:0/2297350822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5324 (secure 0 0 0) 0x7f3ab809bd00 con 0x7f3abc103480 2024-04-26T18:23:53.528 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 92 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-26T18:12:54.869827+0000 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-26T18:22:59.003338+0000 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-04-26T18:23:53.529 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/580247654,v1:172.21.15.37:6803/580247654] [v2:172.21.15.37:6804/580247654,v1:172.21.15.37:6805/580247654] exists,up 6c564014-c9c6-4050-8c9e-ab1b1825a7cb 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/634623221,v1:172.21.15.120:6801/634623221] [v2:172.21.15.120:6802/634623221,v1:172.21.15.120:6803/634623221] exists,up 44d22495-dc0e-4795-9b4c-f9f1d6c4546a 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/1287805124,v1:172.21.15.170:6801/1287805124] [v2:172.21.15.170:6802/1287805124,v1:172.21.15.170:6803/1287805124] exists,up 5b1c3764-35d5-4229-b362-722dbbe07c44 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/3804676062,v1:172.21.15.37:6811/3804676062] [v2:172.21.15.37:6812/3804676062,v1:172.21.15.37:6813/3804676062] exists,up 26ddcdf7-9137-4384-9228-8537b2009989 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/4165015053,v1:172.21.15.120:6809/4165015053] [v2:172.21.15.120:6810/4165015053,v1:172.21.15.120:6811/4165015053] exists,up 633a4362-a88a-4725-b10e-6c893bacd0ee 2024-04-26T18:23:53.530 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/3652828620,v1:172.21.15.170:6809/3652828620] [v2:172.21.15.170:6810/3652828620,v1:172.21.15.170:6811/3652828620] exists,up 493dab6b-f382-47e3-9a56-480a1328eda1 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/1123986561,v1:172.21.15.37:6819/1123986561] [v2:172.21.15.37:6820/1123986561,v1:172.21.15.37:6821/1123986561] exists,up 630f5f5f-942b-4a8c-be69-8b1f55491a3d 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/152962141,v1:172.21.15.120:6817/152962141] [v2:172.21.15.120:6818/152962141,v1:172.21.15.120:6819/152962141] exists,up 78b9cc68-5517-46de-858e-342ba834c2f8 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/428251464,v1:172.21.15.170:6817/428251464] [v2:172.21.15.170:6818/428251464,v1:172.21.15.170:6819/428251464] exists,up 8988738f-786a-4a7e-90da-7bab9c98319f 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/4227134134,v1:172.21.15.37:6827/4227134134] [v2:172.21.15.37:6828/4227134134,v1:172.21.15.37:6829/4227134134] exists,up c35fc73f-65b6-4db3-8180-ee78649aab29 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/2391866955,v1:172.21.15.120:6825/2391866955] [v2:172.21.15.120:6826/2391866955,v1:172.21.15.120:6827/2391866955] exists,up 416522bb-db89-4bfd-83ca-082a3abd1dcf 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 80 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/49311375,v1:172.21.15.170:6825/49311375] [v2:172.21.15.170:6826/49311375,v1:172.21.15.170:6827/49311375] exists,up 54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.c [2,4] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1f [8,1] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.11 [5,9] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.13 [4,9] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.21 [4,8] 2024-04-26T18:23:53.531 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.29 [3,9] 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.31 [4,7] 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.37 [5,6] 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1399831496 expires 2024-04-27T18:13:56.574428+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/1233564213 expires 2024-04-27T18:13:56.574428+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/1233564213 expires 2024-04-27T18:13:56.574428+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/199318008 expires 2024-04-27T18:13:29.830032+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3928213229 expires 2024-04-27T18:13:56.574428+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/53636636 expires 2024-04-27T18:13:29.830032+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1726511030 expires 2024-04-27T18:13:29.830032+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/776601124 expires 2024-04-27T18:13:29.830032+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/155804720 expires 2024-04-27T18:13:56.574428+0000 2024-04-26T18:23:53.532 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/199318008 expires 2024-04-27T18:13:29.830032+0000 2024-04-26T18:23:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 msgr2=0x7f3aa80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 0x7f3aa80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009990 tx=0x7f3aac00b870 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 msgr2=0x7f3abc0771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800d980 tx=0x7f3ab800dc90 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 shutdown_connections 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f3aa80a3c40 0x7f3aa80a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3abc105020 0x7f3abc075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3abc103480 0x7f3abc0771c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 --2- 172.21.15.37:0/2297350822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3abc104680 0x7f3abc075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 >> 172.21.15.37:0/2297350822 conn(0x7f3abc0fea20 msgr2=0x7f3abc107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 shutdown_connections 2024-04-26T18:23:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:53.531+0000 7f3ac3a1a700 1 -- 172.21.15.37:0/2297350822 wait complete. 2024-04-26T18:23:54.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:54 smithi037 ceph-mon[29171]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:54.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:54 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2297350822' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:23:54.288 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T18:23:54.298 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:54 smithi120 ceph-mon[32244]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:54.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:54 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2297350822' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:23:54.553 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:54 smithi170 ceph-mon[32697]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:23:54.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:54 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2297350822' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:23:56.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 -- 172.21.15.37:0/1158461686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0618103e60 msgr2=0x7f06181102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:56.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 --2- 172.21.15.37:0/1158461686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0618103e60 0x7f06181102b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f061000b0a0 tx=0x7f061000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 -- 172.21.15.37:0/1158461686 shutdown_connections 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 --2- 172.21.15.37:0/1158461686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0618103e60 0x7f06181102b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 --2- 172.21.15.37:0/1158461686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06181094e0 0x7f06181098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 --2- 172.21.15.37:0/1158461686 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181034a0 0x7f0618103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 -- 172.21.15.37:0/1158461686 >> 172.21.15.37:0/1158461686 conn(0x7f061807a590 msgr2=0x7f061807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 -- 172.21.15.37:0/1158461686 shutdown_connections 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.195+0000 7f061df1f700 1 -- 172.21.15.37:0/1158461686 wait complete. 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.196+0000 7f061df1f700 1 Processor -- start 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.196+0000 7f061df1f700 1 -- start start 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0618103e60 0x7f061819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181094e0 0x7f06181a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113700 con 0x7f06181034a0 2024-04-26T18:23:56.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113580 con 0x7f06181094e0 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0618113880 con 0x7f0618103e60 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37524/0 (socket says 172.21.15.37:37524) 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 -- 172.21.15.37:0/2028466230 learned_addr learned my addr 172.21.15.37:0/2028466230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f0617fff700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181094e0 0x7f06181a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 -- 172.21.15.37:0/2028466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0618103e60 msgr2=0x7f061819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0618103e60 0x7f061819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 -- 172.21.15.37:0/2028466230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181094e0 msgr2=0x7f06181a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181094e0 0x7f06181a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 -- 172.21.15.37:0/2028466230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06181a0e60 con 0x7f06181034a0 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f06177fe700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f060800b550 tx=0x7f060800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:56.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608015020 con 0x7f06181034a0 2024-04-26T18:23:56.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0608011ce0 con 0x7f06181034a0 2024-04-26T18:23:56.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06181a1150 con 0x7f06181034a0 2024-04-26T18:23:56.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.197+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608010d30 con 0x7f06181034a0 2024-04-26T18:23:56.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.198+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06181a89d0 con 0x7f06181034a0 2024-04-26T18:23:56.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.199+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0618104b80 con 0x7f06181034a0 2024-04-26T18:23:56.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.200+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0608018020 con 0x7f06181034a0 2024-04-26T18:23:56.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.200+0000 7f0614ff9700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 0x7f06000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:56.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.200+0000 7f0616ffd700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 0x7f06000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:56.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.200+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f06080cfb50 con 0x7f06181034a0 2024-04-26T18:23:56.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.201+0000 7f0616ffd700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 0x7f06000a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f061807ae00 tx=0x7f060c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:56.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.205+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f060801f020 con 0x7f06181034a0 2024-04-26T18:23:56.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.460+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f061819d1b0 con 0x7f06181034a0 2024-04-26T18:23:56.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:56 smithi037 ceph-mon[29171]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:56.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.461+0000 7f0614ff9700 1 -- 172.21.15.37:0/2028466230 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7f060809c500 con 0x7f06181034a0 2024-04-26T18:23:56.462 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:56.462 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 msgr2=0x7f06000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 0x7f06000a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f061807ae00 tx=0x7f060c006d20 comp rx=0 tx=0).stop 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 msgr2=0x7f061819be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f060800b550 tx=0x7f060800b860 comp rx=0 tx=0).stop 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 shutdown_connections 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.464+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f06000a3cc0 0x7f06000a6180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06181094e0 0x7f06181a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06181034a0 0x7f061819be50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 --2- 172.21.15.37:0/2028466230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0618103e60 0x7f061819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:56.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 >> 172.21.15.37:0/2028466230 conn(0x7f061807a590 msgr2=0x7f061810e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:56.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 shutdown_connections 2024-04-26T18:23:56.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:56.465+0000 7f061df1f700 1 -- 172.21.15.37:0/2028466230 wait complete. 2024-04-26T18:23:56.466 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:23:56.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:56 smithi120 ceph-mon[32244]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:56.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:56 smithi170 ceph-mon[32697]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:23:57.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:57 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2028466230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:57.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:57 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2028466230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:57 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2028466230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:57.621 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:58.420 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:23:58 smithi037 ceph-mon[29171]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:58.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:23:58 smithi120 ceph-mon[32244]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:58.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:23:58 smithi170 ceph-mon[32697]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 -- 172.21.15.37:0/3940722867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4108500 msgr2=0x7f10c41088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 --2- 172.21.15.37:0/3940722867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4108500 0x7f10c41088e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009990 tx=0x7f10b4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 -- 172.21.15.37:0/3940722867 shutdown_connections 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 --2- 172.21.15.37:0/3940722867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c41027a0 0x7f10c4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 --2- 172.21.15.37:0/3940722867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c4108500 0x7f10c41088e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 --2- 172.21.15.37:0/3940722867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.220+0000 7f10c9441700 1 -- 172.21.15.37:0/3940722867 >> 172.21.15.37:0/3940722867 conn(0x7f10c4077640 msgr2=0x7f10c4077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 -- 172.21.15.37:0/3940722867 shutdown_connections 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 -- 172.21.15.37:0/3940722867 wait complete. 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 Processor -- start 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 -- start start 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 0x7f10c4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.221+0000 7f10c9441700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4113670 con 0x7f10c41027a0 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c9441700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41134f0 con 0x7f10c4101de0 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c9441700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41137f0 con 0x7f10c4108500 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37528/0 (socket says 172.21.15.37:37528) 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 -- 172.21.15.37:0/1858317044 learned_addr learned my addr 172.21.15.37:0/1858317044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c2ffd700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:59.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c37fe700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 0x7f10c4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 -- 172.21.15.37:0/1858317044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 msgr2=0x7f10c4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 0x7f10c4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 -- 172.21.15.37:0/1858317044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 msgr2=0x7f10c419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 -- 172.21.15.37:0/1858317044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c419a090 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c2ffd700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c37fe700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 0x7f10c4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c27fc700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c920 tx=0x7f10b800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8007b00 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b8004ce0 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c419a380 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8005650 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.222+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c41a6640 con 0x7f10c41027a0 2024-04-26T18:23:59.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.223+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c406a110 con 0x7f10c41027a0 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.225+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f10b8004750 con 0x7f10c41027a0 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.225+0000 7f10abfff700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 0x7f10ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.225+0000 7f10c2ffd700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 0x7f10ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.225+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f10b80d0450 con 0x7f10c41027a0 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.226+0000 7f10c2ffd700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 0x7f10ac0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006010 tx=0x7f10b400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:59.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.230+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10b8014030 con 0x7f10c41027a0 2024-04-26T18:23:59.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.481+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c402e490 con 0x7f10c41027a0 2024-04-26T18:23:59.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.482+0000 7f10abfff700 1 -- 172.21.15.37:0/1858317044 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11883 (secure 0 0 0) 0x7f10b809ce00 con 0x7f10c41027a0 2024-04-26T18:23:59.482 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:23:59.483 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:21:29.674118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:23:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 msgr2=0x7f10ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 0x7f10ac0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006010 tx=0x7f10b400b540 comp rx=0 tx=0).stop 2024-04-26T18:23:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 msgr2=0x7f10c419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f10b800c920 tx=0x7f10b800cce0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 shutdown_connections 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f10ac0a3c40 0x7f10ac0a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10c4108500 0x7f10c4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10c4101de0 0x7f10c419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 --2- 172.21.15.37:0/1858317044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10c41027a0 0x7f10c419fcb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 >> 172.21.15.37:0/1858317044 conn(0x7f10c4077640 msgr2=0x7f10c41008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 shutdown_connections 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:23:59.485+0000 7f10c9441700 1 -- 172.21.15.37:0/1858317044 wait complete. 2024-04-26T18:23:59.485 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-26T18:24:00.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:24:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:00 smithi037 ceph-mon[29171]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:00.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:00 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1858317044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:00.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:00 smithi120 ceph-mon[32244]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:00.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:00 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1858317044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:00.520 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:00.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:00 smithi170 ceph-mon[32697]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:00.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:00 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1858317044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:02.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 -- 172.21.15.37:0/4271502222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec102670 msgr2=0x7f92ec102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 --2- 172.21.15.37:0/4271502222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec102670 0x7f92ec102af0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92e00099c0 tx=0x7f92e0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 -- 172.21.15.37:0/4271502222 shutdown_connections 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 --2- 172.21.15.37:0/4271502222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 0x7f92ec10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 --2- 172.21.15.37:0/4271502222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec1073a0 0x7f92ec107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 --2- 172.21.15.37:0/4271502222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec102670 0x7f92ec102af0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 -- 172.21.15.37:0/4271502222 >> 172.21.15.37:0/4271502222 conn(0x7f92ec078190 msgr2=0x7f92ec0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:02.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 -- 172.21.15.37:0/4271502222 shutdown_connections 2024-04-26T18:24:02.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:02 smithi037 ceph-mon[29171]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.280+0000 7f92f2776700 1 -- 172.21.15.37:0/4271502222 wait complete. 2024-04-26T18:24:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 Processor -- start 2024-04-26T18:24:02.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 -- start start 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec102670 0x7f92ec19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 0x7f92ec19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec112570 con 0x7f92ec1073a0 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec1123f0 con 0x7f92ec103030 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.281+0000 7f92f2776700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec1126f0 con 0x7f92ec102670 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37550/0 (socket says 172.21.15.37:37550) 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 -- 172.21.15.37:0/905965568 learned_addr learned my addr 172.21.15.37:0/905965568 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92eb7fe700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 0x7f92ec19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92ebfff700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec102670 0x7f92ec19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 -- 172.21.15.37:0/905965568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec102670 msgr2=0x7f92ec19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:02.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec102670 0x7f92ec19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 -- 172.21.15.37:0/905965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 msgr2=0x7f92ec19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 0x7f92ec19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 -- 172.21.15.37:0/905965568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1a5f70 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f0d13700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92e400ada0 tx=0x7f92e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4017020 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e400ce30 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec1a6140 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005350 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.282+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1a6600 con 0x7f92ec1073a0 2024-04-26T18:24:02.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.283+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec103a90 con 0x7f92ec1073a0 2024-04-26T18:24:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.284+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92e400c980 con 0x7f92ec1073a0 2024-04-26T18:24:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.285+0000 7f92e97fa700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 0x7f92d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.285+0000 7f92ebfff700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 0x7f92d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.285+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f92e40d0160 con 0x7f92ec1073a0 2024-04-26T18:24:02.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.286+0000 7f92ebfff700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 0x7f92d40a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00bbb0 tx=0x7f92dc00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:02.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.289+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92e4021020 con 0x7f92ec1073a0 2024-04-26T18:24:02.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.499+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92ec02e490 con 0x7f92ec1073a0 2024-04-26T18:24:02.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.500+0000 7f92e97fa700 1 -- 172.21.15.37:0/905965568 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22725 (secure 0 0 0) 0x7f92e401a310 con 0x7f92ec1073a0 2024-04-26T18:24:02.501 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:02.501 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":92,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:22:59.003338+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:24:02.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:02 smithi120 ceph-mon[32244]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.502+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 msgr2=0x7f92d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.502+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 0x7f92d40a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00bbb0 tx=0x7f92dc00b590 comp rx=0 tx=0).stop 2024-04-26T18:24:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.502+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 msgr2=0x7f92ec1a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.502+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92e400ada0 tx=0x7f92e400c330 comp rx=0 tx=0).stop 2024-04-26T18:24:02.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 shutdown_connections 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f92d40a3c40 0x7f92d40a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ec1073a0 0x7f92ec1a5a30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92ec102670 0x7f92ec19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 --2- 172.21.15.37:0/905965568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92ec103030 0x7f92ec19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 >> 172.21.15.37:0/905965568 conn(0x7f92ec078190 msgr2=0x7f92ec100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 shutdown_connections 2024-04-26T18:24:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:02.503+0000 7f92f2776700 1 -- 172.21.15.37:0/905965568 wait complete. 2024-04-26T18:24:02.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:02 smithi170 ceph-mon[32697]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:03.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:21:10.996859+0000', 'modified': '2024-04-26T18:21:29.674118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24464, 'mds_1': 24449, 'mds_2': 24442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/707687219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 707687219}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 707687219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/3605086601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3605086601}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 3605086601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/2099915678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2099915678}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2099915678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6839/1569501885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1569501885}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1569501885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.120:6837/1266100832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1266100832}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1266100832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.120:6839/1246417570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1246417570}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1246417570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24464, 24442, 24449]} 2024-04-26T18:24:03.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:24:03.298 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:24:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:03 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/905965568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:03 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/905965568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:03 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/905965568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:04.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:04 smithi037 ceph-mon[29171]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:04 smithi120 ceph-mon[32244]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:04.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:04 smithi170 ceph-mon[32697]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 -- 172.21.15.37:0/281891285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858103e60 msgr2=0x7f68581102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 --2- 172.21.15.37:0/281891285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858103e60 0x7f68581102b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f685000b0a0 tx=0x7f685000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 -- 172.21.15.37:0/281891285 shutdown_connections 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 --2- 172.21.15.37:0/281891285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6858103e60 0x7f68581102b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 --2- 172.21.15.37:0/281891285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 --2- 172.21.15.37:0/281891285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68581034a0 0x7f6858103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 -- 172.21.15.37:0/281891285 >> 172.21.15.37:0/281891285 conn(0x7f685807a590 msgr2=0x7f685807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 -- 172.21.15.37:0/281891285 shutdown_connections 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 -- 172.21.15.37:0/281891285 wait complete. 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.210+0000 7f68603c1700 1 Processor -- start 2024-04-26T18:24:05.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 -- start start 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858103e60 0x7f685819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6858113690 con 0x7f68581034a0 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6858113510 con 0x7f68581094e0 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f68603c1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6858113810 con 0x7f6858103e60 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56672/0 (socket says 172.21.15.37:56672) 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 -- 172.21.15.37:0/866569243 learned_addr learned my addr 172.21.15.37:0/866569243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e95e700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685d95c700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858103e60 0x7f685819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 -- 172.21.15.37:0/866569243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858103e60 msgr2=0x7f685819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858103e60 0x7f685819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 -- 172.21.15.37:0/866569243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 msgr2=0x7f68581999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 -- 172.21.15.37:0/866569243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685819a110 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e95e700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.211+0000 7f685e15d700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f684800b530 tx=0x7f684800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.212+0000 7f684f7fe700 1 -- 172.21.15.37:0/866569243 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848015020 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.212+0000 7f684f7fe700 1 -- 172.21.15.37:0/866569243 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6848011ce0 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.212+0000 7f684f7fe700 1 -- 172.21.15.37:0/866569243 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848010d20 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.212+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685819a400 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.212+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581a66c0 con 0x7f68581034a0 2024-04-26T18:24:05.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.215+0000 7f684f7fe700 1 -- 172.21.15.37:0/866569243 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6848018020 con 0x7f68581034a0 2024-04-26T18:24:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.215+0000 7f686ac53b80 1 Processor -- start 2024-04-26T18:24:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.215+0000 7f686ac53b80 1 -- start start 2024-04-26T18:24:05.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f684f7fe700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 0x7f68440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be75fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 0x561be7608770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561be754e7e0 con 0x561be75fc6f0 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 0x7f68440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561be74b48e0 con 0x561be7601a20 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f686ac53b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561be754a7a0 con 0x561be75fbd50 2024-04-26T18:24:05.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f684f7fe700 1 -- 172.21.15.37:0/866569243 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f68480cfc90 con 0x7f68581034a0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 0x561be7608770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e15d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e15d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:46738/0 (socket says 172.21.15.37:46738) 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e15d700 1 -- 172.21.15.37:0/935186541 learned_addr learned my addr 172.21.15.37:0/935186541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685d95c700 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be75fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 0x7f68440a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6850009e60 tx=0x7f6850004b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 -- 172.21.15.37:0/935186541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 msgr2=0x561be75fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be75fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 -- 172.21.15.37:0/935186541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 msgr2=0x561be7601e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 -- 172.21.15.37:0/935186541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be749b880 con 0x561be75fc6f0 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e15d700 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:05.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.216+0000 7f685e95e700 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 0x561be7608770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f685002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea00dc58152dc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.217+0000 7f684cd70700 1 -- 172.21.15.37:0/935186541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850024230 con 0x561be75fc6f0 2024-04-26T18:24:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.217+0000 7f684cd70700 1 -- 172.21.15.37:0/935186541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6850022c90 con 0x561be75fc6f0 2024-04-26T18:24:05.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.217+0000 7f684cd70700 1 -- 172.21.15.37:0/935186541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850007880 con 0x561be75fc6f0 2024-04-26T18:24:05.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- 172.21.15.37:0/935186541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 msgr2=0x561be7608770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 0x561be7608770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6850009fd0 tx=0x7f685002c000 comp rx=0 tx=0).stop 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- 172.21.15.37:0/935186541 shutdown_connections 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fc6f0 0x561be7608770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be7601e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 --2- 172.21.15.37:0/935186541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be75fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- 172.21.15.37:0/935186541 >> 172.21.15.37:0/935186541 conn(0x561be752a570 msgr2=0x561be75f6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- 172.21.15.37:0/935186541 shutdown_connections 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- 172.21.15.37:0/935186541 wait complete. 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 -- start start 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.218+0000 7f686ac53b80 1 Processor -- start 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be7608380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be7605720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561be7608cb0 con 0x561be7601a20 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561be7508a30 con 0x561be75fc6f0 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561be7608e30 con 0x561be75fbd50 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f686ac53b80 10 client.0 _ll_register_callbacks cb 0x561be74e6900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56690/0 (socket says 172.21.15.37:56690) 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 -- 172.21.15.37:0/2391336874 learned_addr learned my addr 172.21.15.37:0/2391336874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685d95c700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be7605720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e15d700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be7608380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 -- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 msgr2=0x561be7608380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be7608380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 -- 172.21.15.37:0/2391336874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 msgr2=0x561be7605720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be7605720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 -- 172.21.15.37:0/2391336874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be750c430 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f685e95e700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6850030040 tx=0x7f6850009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e94c1ce370a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.219+0000 7f6841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500242b0 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f6850024450 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6850022c90 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68500074e0 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 10 client.15060 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561be7608cb0 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6850022c90 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6850011070 con 0x561be7601a20 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 20 client.15060 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be7615520 0x561be7616990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 4 client.15060 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.220+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be755c1c0 con 0x561be7615520 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.221+0000 7f685e15d700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be7615520 0x561be7616990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.221+0000 7f685e15d700 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be7615520 0x561be7616990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c5acea60a0002e7 server_cookie=9f606a0d6825ff6d in_seq=0 out_seq=0 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.221+0000 7f6841ffb700 10 client.15060 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f6841ffb700 1 -- 172.21.15.37:0/2391336874 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561be755c1c0 con 0x561be7615520 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f6841ffb700 10 client.15060 handle_command_reply: tid=0 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 client.15060 shutdown 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 msgr2=0x561be7605c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6850030040 tx=0x7f6850009570 comp rx=0 tx=0).stop 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 shutdown_connections 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be7601a20 0x561be7605c60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fbd50 0x561be7608380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be7615520 0x561be7616990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 --2- 172.21.15.37:0/2391336874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be7605720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f6841ffb700 0 client.15060 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 >> 172.21.15.37:0/2391336874 conn(0x561be752a570 msgr2=0x561be75fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 shutdown_connections 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 1 -- 172.21.15.37:0/2391336874 wait complete. 2024-04-26T18:24:05.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.222+0000 7f686ac53b80 20 client.15060 trim_cache size 0 max 16384 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 Processor -- start 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 -- start start 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fc6f0 0x561be75fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be75fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561be7614cd0 con 0x561be75fbd50 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561be76153a0 con 0x561be7601a20 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f686ac53b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561be7615a70 con 0x561be75fc6f0 2024-04-26T18:24:05.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56698/0 (socket says 172.21.15.37:56698) 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 -- 172.21.15.37:0/3531371364 learned_addr learned my addr 172.21.15.37:0/3531371364 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685d95c700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fc6f0 0x561be75fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e95e700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be75fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 -- 172.21.15.37:0/3531371364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fc6f0 msgr2=0x561be75fb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fc6f0 0x561be75fb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 -- 172.21.15.37:0/3531371364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 msgr2=0x561be75fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be75fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 -- 172.21.15.37:0/3531371364 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be75f31f0 con 0x561be75fbd50 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e95e700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be75fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.243+0000 7f685e15d700 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f684801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a95f39c2baafd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f6843fff700 1 -- 172.21.15.37:0/3531371364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480a4bb0 con 0x561be75fbd50 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f6843fff700 1 -- 172.21.15.37:0/3531371364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68480b5070 con 0x561be75fbd50 2024-04-26T18:24:05.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f6843fff700 1 -- 172.21.15.37:0/3531371364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684809c400 con 0x561be75fbd50 2024-04-26T18:24:05.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 -- 172.21.15.37:0/3531371364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 msgr2=0x561be75fad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f684801c040 comp rx=0 tx=0).stop 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 -- 172.21.15.37:0/3531371364 shutdown_connections 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be7601a20 0x561be75fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be75fad40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3531371364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be75fc6f0 0x561be75fb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 -- 172.21.15.37:0/3531371364 >> 172.21.15.37:0/3531371364 conn(0x561be752a570 msgr2=0x561be7614570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 -- 172.21.15.37:0/3531371364 shutdown_connections 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.244+0000 7f686ac53b80 1 -- 172.21.15.37:0/3531371364 wait complete. 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 -- start start 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 Processor -- start 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be761c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be7601a20 0x561be7675620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561be7616810 con 0x561be75fbd50 2024-04-26T18:24:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561be7616140 con 0x561be75fc6f0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561be7616ee0 con 0x561be7601a20 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f686ac53b80 10 client.0 _ll_register_callbacks cb 0x561be74e6900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f685e15d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56706/0 (socket says 172.21.15.37:56706) 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f685e15d700 1 -- 172.21.15.37:0/3714031609 learned_addr learned my addr 172.21.15.37:0/3714031609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f685e95e700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be7601a20 0x561be7675620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.245+0000 7f685d95c700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be761c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 -- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be7601a20 msgr2=0x561be7675620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be7601a20 0x561be7675620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 -- 172.21.15.37:0/3714031609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 msgr2=0x561be761c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be761c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 -- 172.21.15.37:0/3714031609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be7675c60 con 0x561be75fbd50 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f685e15d700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68480cdad0 tx=0x7f6848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2877ee8c91bd86ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848051260 con 0x561be75fbd50 2024-04-26T18:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f684809a240 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68480b5070 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68480515e0 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f686ac53b80 10 client.15066 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561be767cf60 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68480b5070 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.246+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f68480b5220 con 0x561be75fbd50 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 20 client.15066 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be76814d0 0x561be76801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 4 client.15066 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "status"}) -- 0x561be7556260 con 0x561be76814d0 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f685d95c700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be76814d0 0x561be76801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f6841ffb700 10 client.15066 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f685d95c700 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be76814d0 0x561be76801e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecd29c19bac5f0ed server_cookie=ca9156ff6d8b44a1 in_seq=0 out_seq=0 2024-04-26T18:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f6841ffb700 1 -- 172.21.15.37:0/3714031609 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561be7556260 con 0x561be76814d0 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f6841ffb700 10 client.15066 handle_command_reply: tid=0 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.247+0000 7f686ac53b80 1 client.15066 shutdown 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 msgr2=0x561be761c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f68480cdad0 tx=0x7f6848004750 comp rx=0 tx=0).stop 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 shutdown_connections 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561be7601a20 0x561be7675620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561be75fbd50 0x561be761c030 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x561be76814d0 0x561be76801e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 --2- 172.21.15.37:0/3714031609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561be75fc6f0 0x561be761c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 >> 172.21.15.37:0/3714031609 conn(0x561be752a570 msgr2=0x561be75fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f6841ffb700 0 client.15066 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 shutdown_connections 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 1 -- 172.21.15.37:0/3714031609 wait complete. 2024-04-26T18:24:05.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f686ac53b80 20 client.15066 trim_cache size 0 max 16384 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24442, 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:05.254 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 160.30695481399999, 2024-04-26T18:24:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:24:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:24:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:24:05.257 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 200.36542450600001 2024-04-26T18:24:05.257 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 msgr2=0x7f68440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 0x7f68440a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6850009e60 tx=0x7f6850004b80 comp rx=0 tx=0).stop 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 msgr2=0x7f685819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f684800b530 tx=0x7f684800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 shutdown_connections 2024-04-26T18:24:05.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f68440a3c40 0x7f68440a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.248+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68581094e0 0x7f68581999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.249+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68581034a0 0x7f685819f7f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.249+0000 7f68603c1700 1 --2- 172.21.15.37:0/866569243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6858103e60 0x7f685819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.249+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 >> 172.21.15.37:0/866569243 conn(0x7f685807a590 msgr2=0x7f6858102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.249+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 shutdown_connections 2024-04-26T18:24:05.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:05.249+0000 7f68603c1700 1 -- 172.21.15.37:0/866569243 wait complete. 2024-04-26T18:24:05.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:24:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:06.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:06 smithi037 ceph-mon[29171]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:24:06.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:06 smithi120 ceph-mon[32244]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:24:06.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:06 smithi170 ceph-mon[32697]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 -- 172.21.15.37:0/174282204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f41094c0 msgr2=0x7f99f41098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/174282204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f41094c0 0x7f99f41098a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009990 tx=0x7f99e4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 -- 172.21.15.37:0/174282204 shutdown_connections 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/174282204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103e30 0x7f99f4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/174282204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f41094c0 0x7f99f41098a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/174282204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f4103470 0x7f99f41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 -- 172.21.15.37:0/174282204 >> 172.21.15.37:0/174282204 conn(0x7f99f407a580 msgr2=0x7f99f407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:07.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 -- 172.21.15.37:0/174282204 shutdown_connections 2024-04-26T18:24:07.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.959+0000 7f99f8fb1700 1 -- 172.21.15.37:0/174282204 wait complete. 2024-04-26T18:24:07.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 Processor -- start 2024-04-26T18:24:07.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 -- start start 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103470 0x7f99f4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f41094c0 0x7f99f40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113640 con 0x7f99f4103e30 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41134c0 con 0x7f99f4103470 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f8fb1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41137c0 con 0x7f99f41094c0 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99ebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99f259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103470 0x7f99f4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99ebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56718/0 (socket says 172.21.15.37:56718) 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.960+0000 7f99ebfff700 1 -- 172.21.15.37:0/752107373 learned_addr learned my addr 172.21.15.37:0/752107373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f41094c0 0x7f99f40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 -- 172.21.15.37:0/752107373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f41094c0 msgr2=0x7f99f40761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f41094c0 0x7f99f40761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 -- 172.21.15.37:0/752107373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103470 msgr2=0x7f99f4075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103470 0x7f99f4075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 -- 172.21.15.37:0/752107373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41a80e0 con 0x7f99f4103e30 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99ebfff700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b530 tx=0x7f99dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99eb7fe700 1 -- 172.21.15.37:0/752107373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc015020 con 0x7f99f4103e30 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99eb7fe700 1 -- 172.21.15.37:0/752107373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99dc011ce0 con 0x7f99f4103e30 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41a83d0 con 0x7f99f4103e30 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99eb7fe700 1 -- 172.21.15.37:0/752107373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc010e40 con 0x7f99f4103e30 2024-04-26T18:24:07.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.961+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a88c0 con 0x7f99f4103e30 2024-04-26T18:24:07.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.963+0000 7f99eb7fe700 1 -- 172.21.15.37:0/752107373 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99dc018020 con 0x7f99f4103e30 2024-04-26T18:24:07.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.964+0000 7f99eb7fe700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 0x7f99e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.964+0000 7f99f259c700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 0x7f99e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.964+0000 7f99eb7fe700 1 -- 172.21.15.37:0/752107373 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f99dc0cfcf0 con 0x7f99f4103e30 2024-04-26T18:24:07.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.964+0000 7f99f259c700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 0x7f99e00a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b5c0 tx=0x7f99e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 Processor -- start 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 -- start start 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f28d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28d6f0 0x561f5f299770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f5f1df7e0 con 0x561f5f292a20 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f5f1458e0 con 0x561f5f28d6f0 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f9a03843b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561f5f1db7a0 con 0x561f5f28cd50 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56728/0 (socket says 172.21.15.37:56728) 2024-04-26T18:24:07.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 -- 172.21.15.37:0/1668526851 learned_addr learned my addr 172.21.15.37:0/1668526851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99ebfff700 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f28d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28d6f0 0x561f5f299770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 -- 172.21.15.37:0/1668526851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 msgr2=0x561f5f28d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f28d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.965+0000 7f99f259c700 1 -- 172.21.15.37:0/1668526851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28d6f0 msgr2=0x561f5f299770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99f259c700 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28d6f0 0x561f5f299770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99f259c700 1 -- 172.21.15.37:0/1668526851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5f12c880 con 0x561f5f292a20 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99f259c700 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99e4015000 tx=0x7f99e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd016b01fc2e575a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99e8ff9700 1 -- 172.21.15.37:0/1668526851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4034070 con 0x561f5f292a20 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99e8ff9700 1 -- 172.21.15.37:0/1668526851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e40249f0 con 0x561f5f292a20 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.966+0000 7f99e8ff9700 1 -- 172.21.15.37:0/1668526851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e402fa30 con 0x561f5f292a20 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- 172.21.15.37:0/1668526851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 msgr2=0x561f5f292e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99e4015000 tx=0x7f99e401f000 comp rx=0 tx=0).stop 2024-04-26T18:24:07.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- 172.21.15.37:0/1668526851 shutdown_connections 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28d6f0 0x561f5f299770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f292e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 --2- 172.21.15.37:0/1668526851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f28d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- 172.21.15.37:0/1668526851 >> 172.21.15.37:0/1668526851 conn(0x561f5f1bb570 msgr2=0x561f5f287900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- 172.21.15.37:0/1668526851 shutdown_connections 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- 172.21.15.37:0/1668526851 wait complete. 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 -- start start 2024-04-26T18:24:07.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.967+0000 7f9a03843b80 1 Processor -- start 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f299290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f292a20 0x561f5f296be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f5f299cb0 con 0x561f5f28d6f0 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f5f199a30 con 0x561f5f292a20 2024-04-26T18:24:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561f5f299e30 con 0x561f5f28cd50 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 10 client.0 _ll_register_callbacks cb 0x561f5f177900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56738/0 (socket says 172.21.15.37:56738) 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 -- 172.21.15.37:0/2711079333 learned_addr learned my addr 172.21.15.37:0/2711079333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f292a20 0x561f5f296be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99f259c700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f299290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 -- 172.21.15.37:0/2711079333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 msgr2=0x561f5f299290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f299290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 -- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f292a20 msgr2=0x561f5f296be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f292a20 0x561f5f296be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 -- 172.21.15.37:0/2711079333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5f19d430 con 0x561f5f28d6f0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99ebfff700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0cd760 tx=0x7f99dc09c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70ce71c640222e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0b4020 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f99dc0a4720 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.968+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e40249f0 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99dc0a4bb0 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 10 client.15078 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f5f2a4470 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e40249f0 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f99dc018c50 con 0x561f5f28d6f0 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 20 client.15078 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f2a6a10 0x561f5f2a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:07.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 4 client.15078 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5f1ed1c0 con 0x561f5f2a6a10 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.969+0000 7f99f259c700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f2a6a10 0x561f5f2a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.970+0000 7f99f259c700 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f2a6a10 0x561f5f2a7e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=914cdd54d57cda28 server_cookie=b0962cf4646cde80 in_seq=0 out_seq=0 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.970+0000 7f99d9ffb700 10 client.15078 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f99d9ffb700 1 -- 172.21.15.37:0/2711079333 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f5f1ed1c0 con 0x561f5f2a6a10 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f99d9ffb700 10 client.15078 handle_command_reply: tid=0 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 client.15078 shutdown 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 msgr2=0x561f5f2966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0cd760 tx=0x7f99dc09c780 comp rx=0 tx=0).stop 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 shutdown_connections 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f2a6a10 0x561f5f2a7e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f292a20 0x561f5f296be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28cd50 0x561f5f299290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f99d9ffb700 0 client.15078 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 --2- 172.21.15.37:0/2711079333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f28d6f0 0x561f5f2966a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 >> 172.21.15.37:0/2711079333 conn(0x561f5f1bb570 msgr2=0x561f5f28bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 shutdown_connections 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 1 -- 172.21.15.37:0/2711079333 wait complete. 2024-04-26T18:24:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.971+0000 7f9a03843b80 20 client.15078 trim_cache size 0 max 16384 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 Processor -- start 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 -- start start 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f296660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f296ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a61c0 con 0x561f5f292a20 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a6890 con 0x561f5f28cd50 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f9a03843b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a6f60 con 0x561f5f28d6f0 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f99f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f99f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56752/0 (socket says 172.21.15.37:56752) 2024-04-26T18:24:08.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.994+0000 7f99f2d9d700 1 -- 172.21.15.37:0/3427603461 learned_addr learned my addr 172.21.15.37:0/3427603461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f259c700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f296660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99ebfff700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f296ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 -- 172.21.15.37:0/3427603461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 msgr2=0x561f5f296ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f296ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 -- 172.21.15.37:0/3427603461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 msgr2=0x561f5f296660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f296660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 -- 172.21.15.37:0/3427603461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5f2a3700 con 0x561f5f292a20 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99ebfff700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f296ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009c10 tx=0x7f99ec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304956e27ca9b9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99dbfff700 1 -- 172.21.15.37:0/3427603461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01b020 con 0x561f5f292a20 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99dbfff700 1 -- 172.21.15.37:0/3427603461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec007cc0 con 0x561f5f292a20 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.995+0000 7f99dbfff700 1 -- 172.21.15.37:0/3427603461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec016700 con 0x561f5f292a20 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- 172.21.15.37:0/3427603461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 msgr2=0x561f5f2874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:08.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009c10 tx=0x7f99ec009040 comp rx=0 tx=0).stop 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- 172.21.15.37:0/3427603461 shutdown_connections 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2874d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f296660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 --2- 172.21.15.37:0/3427603461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f296ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- 172.21.15.37:0/3427603461 >> 172.21.15.37:0/3427603461 conn(0x561f5f1bb570 msgr2=0x561f5f295c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- 172.21.15.37:0/3427603461 shutdown_connections 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- 172.21.15.37:0/3427603461 wait complete. 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 -- start start 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.996+0000 7f9a03843b80 1 Processor -- start 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f2b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f2b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a7d00 con 0x561f5f292a20 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a7630 con 0x561f5f28cd50 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561f5f2a83d0 con 0x561f5f28d6f0 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 10 client.0 _ll_register_callbacks cb 0x561f5f177900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f2b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56758/0 (socket says 172.21.15.37:56758) 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 -- 172.21.15.37:0/893419455 learned_addr learned my addr 172.21.15.37:0/893419455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f259c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f2b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:46826/0 (socket says 172.21.15.37:46826) 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99ebfff700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f2b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 -- 172.21.15.37:0/893419455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 msgr2=0x561f5f2b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f2b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 -- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 msgr2=0x561f5f2b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:08.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f2b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 -- 172.21.15.37:0/893419455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5f2b0ee0 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99f2d9d700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0077e0 tx=0x7f99ec017720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cabec78e3d8ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec017910 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec007cc0 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.997+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f99ec003a00 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99ec030030 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f5f28f110 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec007cc0 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f99ec005040 con 0x561f5f292a20 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:08.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 20 client.15084 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f313220 0x561f5f311f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 4 client.15084 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "status"}) -- 0x561f5f1e7260 con 0x561f5f313220 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.998+0000 7f99f259c700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f313220 0x561f5f311f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f99d9ffb700 10 client.15084 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f99f259c700 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f313220 0x561f5f311f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96e40bb14fe99c21 server_cookie=c20d5836ec02baca in_seq=0 out_seq=0 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f99d9ffb700 1 -- 172.21.15.37:0/893419455 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f5f1e7260 con 0x561f5f313220 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f99d9ffb700 10 client.15084 handle_command_reply: tid=0 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 client.15084 shutdown 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 msgr2=0x561f5f2b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0077e0 tx=0x7f99ec017720 comp rx=0 tx=0).stop 2024-04-26T18:24:08.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 shutdown_connections 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x561f5f313220 0x561f5f311f30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f5f292a20 0x561f5f2b0830 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f5f28cd50 0x561f5f2b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 --2- 172.21.15.37:0/893419455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f5f28d6f0 0x561f5f2b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 >> 172.21.15.37:0/893419455 conn(0x561f5f1bb570 msgr2=0x561f5f294f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f99d9ffb700 0 client.15084 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 shutdown_connections 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 1 -- 172.21.15.37:0/893419455 wait complete. 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:07.999+0000 7f9a03843b80 20 client.15084 trim_cache size 0 max 16384 2024-04-26T18:24:08.006 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 164.06934791099999, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 205.70492725899999 2024-04-26T18:24:08.007 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 msgr2=0x7f99e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 0x7f99e00a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b5c0 tx=0x7f99e4005960 comp rx=0 tx=0).stop 2024-04-26T18:24:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 msgr2=0x7f99f4075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b530 tx=0x7f99dc00b840 comp rx=0 tx=0).stop 2024-04-26T18:24:08.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 shutdown_connections 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f99e00a3c40 0x7f99e00a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99f41094c0 0x7f99f40761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99f4103470 0x7f99f4075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 --2- 172.21.15.37:0/752107373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103e30 0x7f99f4075c60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 >> 172.21.15.37:0/752107373 conn(0x7f99f407a580 msgr2=0x7f99f4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 shutdown_connections 2024-04-26T18:24:08.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:08.000+0000 7f99f8fb1700 1 -- 172.21.15.37:0/752107373 wait complete. 2024-04-26T18:24:08.323 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:08 smithi037 ceph-mon[29171]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:08.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:08 smithi120 ceph-mon[32244]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:08.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:08 smithi170 ceph-mon[32697]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:08.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:24:09.030 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:10.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:10 smithi037 ceph-mon[29171]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:10 smithi120 ceph-mon[32244]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:10.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:10 smithi170 ceph-mon[32697]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 -- 172.21.15.37:0/3650875621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e701027a0 msgr2=0x7f7e7010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/3650875621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e701027a0 0x7f7e7010e9b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b0a0 tx=0x7f7e6800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 -- 172.21.15.37:0/3650875621 shutdown_connections 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/3650875621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e701027a0 0x7f7e7010e9b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/3650875621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e70107c20 0x7f7e70108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/3650875621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 0x7f7e70102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 -- 172.21.15.37:0/3650875621 >> 172.21.15.37:0/3650875621 conn(0x7f7e70077640 msgr2=0x7f7e70077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 -- 172.21.15.37:0/3650875621 shutdown_connections 2024-04-26T18:24:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 -- 172.21.15.37:0/3650875621 wait complete. 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.675+0000 7f7e77bc5700 1 Processor -- start 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 -- start start 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 0x7f7e70103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e701027a0 0x7f7e70103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70111d60 con 0x7f7e70107c20 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70111be0 con 0x7f7e701027a0 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e77bc5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70111ee0 con 0x7f7e70101de0 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56774/0 (socket says 172.21.15.37:56774) 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 -- 172.21.15.37:0/786118782 learned_addr learned my addr 172.21.15.37:0/786118782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e75160700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e701027a0 0x7f7e70103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e75961700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 0x7f7e70103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 -- 172.21.15.37:0/786118782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 msgr2=0x7f7e70103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 0x7f7e70103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 -- 172.21.15.37:0/786118782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e701027a0 msgr2=0x7f7e70103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e701027a0 0x7f7e70103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.676+0000 7f7e76162700 1 -- 172.21.15.37:0/786118782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701a7f60 con 0x7f7e70107c20 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e76162700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b070 tx=0x7f7e6800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e66ffd700 1 -- 172.21.15.37:0/786118782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68021070 con 0x7f7e70107c20 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e66ffd700 1 -- 172.21.15.37:0/786118782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e68007c80 con 0x7f7e70107c20 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a81f0 con 0x7f7e70107c20 2024-04-26T18:24:10.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e66ffd700 1 -- 172.21.15.37:0/786118782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68005440 con 0x7f7e70107c20 2024-04-26T18:24:10.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.677+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a86a0 con 0x7f7e70107c20 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.679+0000 7f7e66ffd700 1 -- 172.21.15.37:0/786118782 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e68024070 con 0x7f7e70107c20 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.680+0000 7f7e66ffd700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 0x7f7e5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.680+0000 7f7e75961700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 0x7f7e5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.680+0000 7f7e66ffd700 1 -- 172.21.15.37:0/786118782 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f7e680d1620 con 0x7f7e70107c20 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.680+0000 7f7e75961700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 0x7f7e5c0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005ea0 tx=0x7f7e6c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 Processor -- start 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 -- start start 2024-04-26T18:24:10.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d2d50 0x55da4c9d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9257e0 con 0x55da4c9d8a20 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da4c88b8e0 con 0x55da4c9d2d50 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e82457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9217a0 con 0x55da4c9d36f0 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56788/0 (socket says 172.21.15.37:56788) 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e75961700 1 -- 172.21.15.37:0/219622234 learned_addr learned my addr 172.21.15.37:0/219622234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e75160700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d2d50 0x55da4c9d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.681+0000 7f7e76162700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 -- 172.21.15.37:0/219622234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 msgr2=0x55da4c9df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 -- 172.21.15.37:0/219622234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d2d50 msgr2=0x55da4c9d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d2d50 0x55da4c9d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 -- 172.21.15.37:0/219622234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4c872880 con 0x55da4c9d8a20 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e76162700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e75961700 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a520 tx=0x7f7e6c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80780a258fe66ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e5bfff700 1 -- 172.21.15.37:0/219622234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c010cd0 con 0x55da4c9d8a20 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e5bfff700 1 -- 172.21.15.37:0/219622234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55da4c9d8a20 2024-04-26T18:24:10.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.682+0000 7f7e5bfff700 1 -- 172.21.15.37:0/219622234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00f6b0 con 0x55da4c9d8a20 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- 172.21.15.37:0/219622234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 msgr2=0x55da4c9d8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a520 tx=0x7f7e6c00af90 comp rx=0 tx=0).stop 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- 172.21.15.37:0/219622234 shutdown_connections 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d8a20 0x55da4c9d8e00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 --2- 172.21.15.37:0/219622234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d2d50 0x55da4c9d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- 172.21.15.37:0/219622234 >> 172.21.15.37:0/219622234 conn(0x55da4c901570 msgr2=0x55da4c9cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- 172.21.15.37:0/219622234 shutdown_connections 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- 172.21.15.37:0/219622234 wait complete. 2024-04-26T18:24:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 -- start start 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.683+0000 7f7e82457b80 1 Processor -- start 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d8a20 0x55da4c9dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9dfcb0 con 0x55da4c9d2d50 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da4c8dfa30 con 0x55da4c9d8a20 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9dfe30 con 0x55da4c9d36f0 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e82457b80 10 client.0 _ll_register_callbacks cb 0x55da4c8bd900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56802/0 (socket says 172.21.15.37:56802) 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 -- 172.21.15.37:0/653454515 learned_addr learned my addr 172.21.15.37:0/653454515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e76162700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d8a20 0x55da4c9dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75160700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 -- 172.21.15.37:0/653454515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 msgr2=0x55da4c9dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 -- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d8a20 msgr2=0x55da4c9dcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d8a20 0x55da4c9dcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 -- 172.21.15.37:0/653454515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4c8e3430 con 0x55da4c9d2d50 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e75961700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c01de40 tx=0x7f7e6c00aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb85fd6fa56f1474 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.684+0000 7f7e597fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c01d6d0 con 0x55da4c9d2d50 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f7e6c01d870 con 0x55da4c9d2d50 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c022070 con 0x55da4c9d2d50 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c01da70 con 0x55da4c9d2d50 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 10 client.15096 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da4c9ea460 con 0x55da4c9d2d50 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55da4c9d2d50 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7e6c004ce0 con 0x55da4c9d2d50 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 20 client.15096 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4c9ec890 0x55da4c9edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 4 client.15096 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.685+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da4c9331c0 con 0x55da4c9ec890 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.686+0000 7f7e75160700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4c9ec890 0x55da4c9edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.686+0000 7f7e75160700 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4c9ec890 0x55da4c9edd00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6c93e1f3133856 server_cookie=94c751c0007545a7 in_seq=0 out_seq=0 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.686+0000 7f7e597fa700 10 client.15096 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e597fa700 1 -- 172.21.15.37:0/653454515 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da4c9331c0 con 0x55da4c9ec890 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e597fa700 10 client.15096 handle_command_reply: tid=0 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 client.15096 shutdown 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 msgr2=0x55da4c9df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c01de40 tx=0x7f7e6c00aa50 comp rx=0 tx=0).stop 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 shutdown_connections 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4c9ec890 0x55da4c9edd00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d8a20 0x55da4c9dcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9df3b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 --2- 172.21.15.37:0/653454515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d36f0 0x55da4c9dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e597fa700 0 client.15096 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 >> 172.21.15.37:0/653454515 conn(0x55da4c901570 msgr2=0x55da4c9d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 shutdown_connections 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 1 -- 172.21.15.37:0/653454515 wait complete. 2024-04-26T18:24:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.687+0000 7f7e82457b80 20 client.15096 trim_cache size 0 max 16384 2024-04-26T18:24:10.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 Processor -- start 2024-04-26T18:24:10.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 -- start start 2024-04-26T18:24:10.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9ec040 con 0x55da4c9d2d50 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9ec710 con 0x55da4c9d36f0 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e82457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9ecde0 con 0x55da4c9d8a20 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56804/0 (socket says 172.21.15.37:56804) 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e75961700 1 -- 172.21.15.37:0/1268666460 learned_addr learned my addr 172.21.15.37:0/1268666460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.708+0000 7f7e75160700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e76162700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 -- 172.21.15.37:0/1268666460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 msgr2=0x55da4c9db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 -- 172.21.15.37:0/1268666460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 msgr2=0x55da4c9db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9db090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 -- 172.21.15.37:0/1268666460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4c9ec530 con 0x55da4c9d2d50 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e76162700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75160700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e75961700 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ecc0 tx=0x7f7e6c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c670d7b51b06693 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e5b7fe700 1 -- 172.21.15.37:0/1268666460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c014040 con 0x55da4c9d2d50 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e5b7fe700 1 -- 172.21.15.37:0/1268666460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55da4c9d2d50 2024-04-26T18:24:10.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.709+0000 7f7e5b7fe700 1 -- 172.21.15.37:0/1268666460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c031830 con 0x55da4c9d2d50 2024-04-26T18:24:10.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- 172.21.15.37:0/1268666460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 msgr2=0x55da4c9ddd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ecc0 tx=0x7f7e6c005fd0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- 172.21.15.37:0/1268666460 shutdown_connections 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9ddd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1268666460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- 172.21.15.37:0/1268666460 >> 172.21.15.37:0/1268666460 conn(0x55da4c901570 msgr2=0x55da4c9cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- 172.21.15.37:0/1268666460 shutdown_connections 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- 172.21.15.37:0/1268666460 wait complete. 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 -- start start 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.710+0000 7f7e82457b80 1 Processor -- start 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9edb80 con 0x55da4c9d2d50 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9ed4b0 con 0x55da4c9d36f0 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da4c9ee250 con 0x55da4c9d8a20 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e82457b80 10 client.0 _ll_register_callbacks cb 0x55da4c8bd900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56820/0 (socket says 172.21.15.37:56820) 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 -- 172.21.15.37:0/1477163214 learned_addr learned my addr 172.21.15.37:0/1477163214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75160700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e76162700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 -- 172.21.15.37:0/1477163214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 msgr2=0x55da4c9f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 -- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 msgr2=0x55da4c9f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75961700 1 -- 172.21.15.37:0/1477163214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4c9f0840 con 0x55da4c9d2d50 2024-04-26T18:24:10.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e76162700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.711+0000 7f7e75160700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e75961700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a520 tx=0x7f7e6c00e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159d55d9c269b6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c014040 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f7e6c01d650 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c022070 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c01d850 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da4ca54350 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c022070 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7e6c044650 con 0x55da4c9d2d50 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:24:10.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.712+0000 7f7e82457b80 20 client.15102 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4ca58b40 0x55da4ca57850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e82457b80 4 client.15102 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "status"}) -- 0x55da4c92d260 con 0x55da4ca58b40 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e75160700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4ca58b40 0x55da4ca57850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e75160700 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4ca58b40 0x55da4ca57850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d700e61fa1f1e server_cookie=bb0f23e451c11a9e in_seq=0 out_seq=0 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e597fa700 10 client.15102 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e597fa700 1 -- 172.21.15.37:0/1477163214 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55da4c92d260 con 0x55da4ca58b40 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e597fa700 10 client.15102 handle_command_reply: tid=0 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.713+0000 7f7e82457b80 1 client.15102 shutdown 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 msgr2=0x55da4c9f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00a520 tx=0x7f7e6c00e920 comp rx=0 tx=0).stop 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 shutdown_connections 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55da4ca58b40 0x55da4ca57850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da4c9d8a20 0x55da4c9f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da4c9d2d50 0x55da4c9f7180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 --2- 172.21.15.37:0/1477163214 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55da4c9d36f0 0x55da4c9f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e597fa700 0 client.15102 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 >> 172.21.15.37:0/1477163214 conn(0x55da4c901570 msgr2=0x55da4c9ddd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 shutdown_connections 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 1 -- 172.21.15.37:0/1477163214 wait complete. 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.714+0000 7f7e82457b80 20 client.15102 trim_cache size 0 max 16384 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:10.719 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24464, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 178.808821395, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 195.18075267500001 2024-04-26T18:24:10.720 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 msgr2=0x7f7e5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 0x7f7e5c0a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005ea0 tx=0x7f7e6c005e30 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 msgr2=0x7f7e701041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800b070 tx=0x7f7e6800ce50 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 shutdown_connections 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7e5c0a3cc0 0x7f7e5c0a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e70107c20 0x7f7e701041c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e70101de0 0x7f7e70103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 --2- 172.21.15.37:0/786118782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e701027a0 0x7f7e70103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 >> 172.21.15.37:0/786118782 conn(0x7f7e70077640 msgr2=0x7f7e701015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:10.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 shutdown_connections 2024-04-26T18:24:10.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:10.715+0000 7f7e77bc5700 1 -- 172.21.15.37:0/786118782 wait complete. 2024-04-26T18:24:11.479 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T18:24:11.488 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T18:24:11.488 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T18:24:11.489 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T18:24:11.489 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T18:24:11.490 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:24:11.490 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:24:11.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T18:24:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.597+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3427997147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4102b30 msgr2=0x7fdfd4102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.597+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3427997147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4102b30 0x7fdfd4102fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02a410 comp rx=0 tx=0).stop 2024-04-26T18:24:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3427997147 shutdown_connections 2024-04-26T18:24:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3427997147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3427997147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4102b30 0x7fdfd4102fb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3427997147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfd4101940 0x7fdfd4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3427997147 >> 172.21.15.37:0/3427997147 conn(0x7fdfd40fceb0 msgr2=0x7fdfd40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3427997147 shutdown_connections 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3427997147 wait complete. 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 Processor -- start 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- start start 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfd4102b30 0x7fdfd4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd41a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410e520 con 0x7fdfd4101940 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410e3a0 con 0x7fdfd4102b30 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd92d1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410e6a0 con 0x7fdfd41034f0 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56832/0 (socket says 172.21.15.37:56832) 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.598+0000 7fdfd2ffd640 1 -- 172.21.15.37:0/3072901222 learned_addr learned my addr 172.21.15.37:0/3072901222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd37fe640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd41a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 -- 172.21.15.37:0/3072901222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 msgr2=0x7fdfd41a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd41a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 -- 172.21.15.37:0/3072901222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfd4102b30 msgr2=0x7fdfd4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfd4102b30 0x7fdfd4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 -- 172.21.15.37:0/3072901222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41a0710 con 0x7fdfd4101940 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd37fe640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd41a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd2ffd640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400d950 tx=0x7fdfc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4017020 con 0x7fdfd4101940 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41a0a00 con 0x7fdfd4101940 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc4009c30 con 0x7fdfd4101940 2024-04-26T18:24:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400f6b0 con 0x7fdfd4101940 2024-04-26T18:24:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.599+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41a0f50 con 0x7fdfd4101940 2024-04-26T18:24:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.600+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4108c60 con 0x7fdfd4101940 2024-04-26T18:24:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.601+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdfc401a020 con 0x7fdfd4101940 2024-04-26T18:24:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.602+0000 7fdfd0ff9640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 0x7fdfc00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.602+0000 7fdfcbfff640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 0x7fdfc00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.602+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7fdfc40d0550 con 0x7fdfd4101940 2024-04-26T18:24:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.603+0000 7fdfcbfff640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 0x7fdfc00a6110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0096c0 tx=0x7fdfbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.606+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfc4009de0 con 0x7fdfd4101940 2024-04-26T18:24:11.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:11.846+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fdfd402dad0 con 0x7fdfd4101940 2024-04-26T18:24:12.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.164+0000 7fdfd0ff9640 1 -- 172.21.15.37:0/3072901222 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fdfc409cea0 con 0x7fdfd4101940 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 msgr2=0x7fdfc00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 0x7fdfc00a6110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0096c0 tx=0x7fdfbc000bc0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 msgr2=0x7fdfd4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400d950 tx=0x7fdfc400de10 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 shutdown_connections 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fdfc00a3c50 0x7fdfc00a6110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdfd4102b30 0x7fdfd4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfd41034f0 0x7fdfd41a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 --2- 172.21.15.37:0/3072901222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4101940 0x7fdfd4198af0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 >> 172.21.15.37:0/3072901222 conn(0x7fdfd40fceb0 msgr2=0x7fdfd4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.167+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 shutdown_connections 2024-04-26T18:24:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.168+0000 7fdfd92d1640 1 -- 172.21.15.37:0/3072901222 wait complete. 2024-04-26T18:24:12.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T18:24:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.296+0000 7f9a8090a640 1 -- 172.21.15.37:0/1803578993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102a80 msgr2=0x7f9a7c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.296+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1803578993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102a80 0x7f9a7c102f00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009900 tx=0x7f9a6402a410 comp rx=0 tx=0).stop 2024-04-26T18:24:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 -- 172.21.15.37:0/1803578993 shutdown_connections 2024-04-26T18:24:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1803578993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c103440 0x7f9a7c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1803578993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c102a80 0x7f9a7c102f00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1803578993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c101890 0x7f9a7c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 -- 172.21.15.37:0/1803578993 >> 172.21.15.37:0/1803578993 conn(0x7f9a7c0fce00 msgr2=0x7f9a7c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 -- 172.21.15.37:0/1803578993 shutdown_connections 2024-04-26T18:24:12.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.297+0000 7f9a8090a640 1 -- 172.21.15.37:0/1803578993 wait complete. 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 Processor -- start 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 -- start start 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 0x7f9a7c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102a80 0x7f9a7c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c10d3f0 con 0x7f9a7c103440 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c10d270 con 0x7f9a7c102a80 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a8090a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c10d570 con 0x7f9a7c101890 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56864/0 (socket says 172.21.15.37:56864) 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 -- 172.21.15.37:0/1492055485 learned_addr learned my addr 172.21.15.37:0/1492055485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7a59c640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 0x7f9a7c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a73fff640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102a80 0x7f9a7c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 -- 172.21.15.37:0/1492055485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 msgr2=0x7f9a7c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 0x7f9a7c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 -- 172.21.15.37:0/1492055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102a80 msgr2=0x7f9a7c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102a80 0x7f9a7c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7ad9d640 1 -- 172.21.15.37:0/1492055485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c19f660 con 0x7f9a7c103440 2024-04-26T18:24:12.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.298+0000 7f9a7a59c640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 0x7f9a7c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a7ad9d640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400eb10 tx=0x7f9a7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74004b20 con 0x7f9a7c103440 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a74004cc0 con 0x7f9a7c103440 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c06a040 con 0x7f9a7c103440 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74010670 con 0x7f9a7c103440 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.299+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c06a510 con 0x7f9a7c103440 2024-04-26T18:24:12.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.300+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c0506c0 con 0x7f9a7c103440 2024-04-26T18:24:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.302+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9a74005020 con 0x7f9a7c103440 2024-04-26T18:24:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.303+0000 7f9a737fe640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 0x7f9a680a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.303+0000 7f9a7a59c640 1 -- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 msgr2=0x7f9a680a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.303+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f9a740d0aa0 con 0x7f9a7c103440 2024-04-26T18:24:12.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.303+0000 7f9a7a59c640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 0x7f9a680a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:24:12.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.308+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a7409d3f0 con 0x7f9a7c103440 2024-04-26T18:24:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-mon[29171]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:24:12.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setuser ceph since I am not root 2024-04-26T18:24:12.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: ignoring --setgroup ceph since I am not root 2024-04-26T18:24:12.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:12.361+0000 7fabde263200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:24:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.496+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9a7c19f830 con 0x7f9a7c103440 2024-04-26T18:24:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.500+0000 7f9a737fe640 1 -- 172.21.15.37:0/1492055485 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f9a7409ccf0 con 0x7f9a7c103440 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-mon[32244]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: ignoring --setuser ceph since I am not root 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: ignoring --setgroup ceph since I am not root 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-mgr[33631]: -- 172.21.15.120:0/3356944922 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c645ac5a0 con 0x560c64580c00 2024-04-26T18:24:12.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:12.351+0000 7fe780cdf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 msgr2=0x7f9a680a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 0x7f9a680a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 msgr2=0x7f9a7c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400eb10 tx=0x7f9a7400c2d0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 shutdown_connections 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f9a680a3ca0 0x7f9a680a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.502+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a7c102a80 0x7f9a7c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.503+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a7c103440 0x7f9a7c19eff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.503+0000 7f9a8090a640 1 --2- 172.21.15.37:0/1492055485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a7c101890 0x7f9a7c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.503+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 >> 172.21.15.37:0/1492055485 conn(0x7f9a7c0fce00 msgr2=0x7f9a7c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.503+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 shutdown_connections 2024-04-26T18:24:12.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.503+0000 7f9a8090a640 1 -- 172.21.15.37:0/1492055485 wait complete. 2024-04-26T18:24:12.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T18:24:12.531 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-mon[32697]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: ignoring --setuser ceph since I am not root 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: ignoring --setgroup ceph since I am not root 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-mgr[34556]: -- 172.21.15.170:0/174262951 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557938de65a0 con 0x557938dbb000 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:12.348+0000 7f8f99990200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:24:12.532 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:12.531+0000 7f8f99990200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:12.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 -- 172.21.15.37:0/2593445274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc103410 msgr2=0x7f6dfc105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2593445274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc103410 0x7f6dfc105800 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009900 tx=0x7f6dec02a410 comp rx=0 tx=0).stop 2024-04-26T18:24:12.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 -- 172.21.15.37:0/2593445274 shutdown_connections 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2593445274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc105d40 0x7f6dfc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2593445274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 0x7f6dfc10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2593445274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc103410 0x7f6dfc105800 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 -- 172.21.15.37:0/2593445274 >> 172.21.15.37:0/2593445274 conn(0x7f6dfc0fce00 msgr2=0x7f6dfc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 -- 172.21.15.37:0/2593445274 shutdown_connections 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 -- 172.21.15.37:0/2593445274 wait complete. 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.619+0000 7f6e038e3640 1 Processor -- start 2024-04-26T18:24:12.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 -- start start 2024-04-26T18:24:12.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 0x7f6dfc1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc10e510 con 0x7f6dfc105d40 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc10e390 con 0x7f6dfc103410 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e038e3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc10e690 con 0x7f6dfc108670 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e0167f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56888/0 (socket says 172.21.15.37:56888) 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e0167f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:46920/0 (socket says 172.21.15.37:46920) 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 -- 172.21.15.37:0/2839669817 learned_addr learned my addr 172.21.15.37:0/2839669817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e01e80640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 0x7f6dfc1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 msgr2=0x7f6dfc1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 0x7f6dfc1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 msgr2=0x7f6dfc198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 -- 172.21.15.37:0/2839669817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc1a0770 con 0x7f6dfc105d40 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e0167f640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.620+0000 7f6e00e7e640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6df800e9c0 tx=0x7f6df800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.621+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df800cd80 con 0x7f6dfc105d40 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.621+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df80054d0 con 0x7f6dfc105d40 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.621+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc1a0a60 con 0x7f6dfc105d40 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.621+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8010430 con 0x7f6dfc105d40 2024-04-26T18:24:12.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.621+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc1a0fb0 con 0x7f6dfc105d40 2024-04-26T18:24:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.622+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc0fe8f0 con 0x7f6dfc105d40 2024-04-26T18:24:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.623+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6df8004fd0 con 0x7f6dfc105d40 2024-04-26T18:24:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.624+0000 7f6df27fc640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 0x7f6de80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.624+0000 7f6e0167f640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 msgr2=0x7f6de80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.624+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f6df80d08e0 con 0x7f6dfc105d40 2024-04-26T18:24:12.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.624+0000 7f6e0167f640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 0x7f6de80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:24:12.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.632+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6df809d230 con 0x7f6dfc105d40 2024-04-26T18:24:12.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.821+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f6dfc0635a0 con 0x7f6dfc105d40 2024-04-26T18:24:12.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.825+0000 7f6e0167f640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 msgr2=0x7f6de80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:12.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.825+0000 7f6e0167f640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 0x7f6de80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:24:12.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.825+0000 7f6df27fc640 1 -- 172.21.15.37:0/2839669817 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f6df809cb30 con 0x7f6dfc105d40 2024-04-26T18:24:12.825 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:12.634+0000 7f8f99990200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 msgr2=0x7f6de80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 0x7f6de80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 msgr2=0x7f6dfc199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6df800e9c0 tx=0x7f6df800ee80 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 shutdown_connections 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f6de80a3ca0 0x7f6de80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dfc108670 0x7f6dfc1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc105d40 0x7f6dfc199090 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 --2- 172.21.15.37:0/2839669817 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6dfc103410 0x7f6dfc198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 >> 172.21.15.37:0/2839669817 conn(0x7f6dfc0fce00 msgr2=0x7f6dfc103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 shutdown_connections 2024-04-26T18:24:12.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:12.827+0000 7f6e038e3640 1 -- 172.21.15.37:0/2839669817 wait complete. 2024-04-26T18:24:12.837 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:12.556+0000 7fabde263200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:12.837 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:12.669+0000 7fabde263200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:24:12.838 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T18:24:12.862 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T18:24:12.862 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T18:24:12.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:12.862 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T18:24:12.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:12.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T18:24:12.944 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-26T18:24:12.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:24:12.946 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T18:24:12.946 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T18:24:12.947 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:12.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T18:24:12.979 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:12.543+0000 7fe780cdf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:12.979 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:12.650+0000 7fe780cdf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:24:12.979 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:12.978+0000 7fe780cdf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:24:13.090 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:12.999+0000 7fabde263200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:24:13.090 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:13.089+0000 7fabde263200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:24:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:13.252 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:13.065+0000 7fe780cdf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:24:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:13 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:24:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:13 smithi120 ceph-mon[32244]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T18:24:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:13 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1492055485' entity='client.admin' 2024-04-26T18:24:13.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:13 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2839669817' entity='client.admin' 2024-04-26T18:24:13.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:12.944+0000 7f8f99990200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:24:13.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:13.029+0000 7f8f99990200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:24:13.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:13 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:24:13.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:13 smithi170 ceph-mon[32697]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T18:24:13.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:13 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1492055485' entity='client.admin' 2024-04-26T18:24:13.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:13 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2839669817' entity='client.admin' 2024-04-26T18:24:13.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3072901222' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:24:13.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-mon[29171]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-26T18:24:13.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1492055485' entity='client.admin' 2024-04-26T18:24:13.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:13 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2839669817' entity='client.admin' 2024-04-26T18:24:14.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:14.245+0000 7fe780cdf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:24:14.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:14.411+0000 7fe780cdf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:14.552 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:14.212+0000 7f8f99990200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:24:14.552 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:14.382+0000 7f8f99990200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:14.708 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:14.368+0000 7fabde263200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:24:14.709 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:14.551+0000 7fabde263200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:14.860 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:14.578+0000 7fe780cdf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:24:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 -- 172.21.15.37:0/3841427465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8109560 msgr2=0x7fb0d8109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/3841427465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8109560 0x7fb0d8109940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40099c0 tx=0x7fb0d4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 -- 172.21.15.37:0/3841427465 shutdown_connections 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/3841427465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8103e80 0x7fb0d8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/3841427465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8109560 0x7fb0d8109940 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/3841427465 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d8103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 -- 172.21.15.37:0/3841427465 >> 172.21.15.37:0/3841427465 conn(0x7fb0d807a5d0 msgr2=0x7fb0d807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 -- 172.21.15.37:0/3841427465 shutdown_connections 2024-04-26T18:24:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.958+0000 7fb0e04dc700 1 -- 172.21.15.37:0/3841427465 wait complete. 2024-04-26T18:24:14.959 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:14.736+0000 7fabde263200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 Processor -- start 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 -- start start 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8109560 0x7fb0d810adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8113640 con 0x7fb0d8103e80 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81134c0 con 0x7fb0d81034c0 2024-04-26T18:24:14.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0e04dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81137c0 con 0x7fb0d8109560 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0dda77700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0de278700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0dda77700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42042/0 (socket says 172.21.15.37:42042) 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0de278700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:60218/0 (socket says 172.21.15.37:60218) 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0dda77700 1 -- 172.21.15.37:0/2696753721 learned_addr learned my addr 172.21.15.37:0/2696753721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.959+0000 7fb0dea79700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8109560 0x7fb0d810adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8109560 msgr2=0x7fb0d810adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8109560 0x7fb0d810adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 msgr2=0x7fb0d810a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 -- 172.21.15.37:0/2696753721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d81ac370 con 0x7fb0d81034c0 2024-04-26T18:24:14.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0dda77700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0de278700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40099c0 tx=0x7fb0d4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4026070 con 0x7fb0d81034c0 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d4004670 con 0x7fb0d81034c0 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d81ac600 con 0x7fb0d81034c0 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.960+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400f960 con 0x7fb0d81034c0 2024-04-26T18:24:14.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.961+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81acab0 con 0x7fb0d81034c0 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.962+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d8104bd0 con 0x7fb0d81034c0 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.963+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb0d4029070 con 0x7fb0d81034c0 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.963+0000 7fb0cf7fe700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 0x7fb0c40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.963+0000 7fb0dda77700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 msgr2=0x7fb0c40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.963+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7fb0d40d0ba0 con 0x7fb0d81034c0 2024-04-26T18:24:14.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.963+0000 7fb0dda77700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 0x7fb0c40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:24:14.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:14.968+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0d405dff0 con 0x7fb0d81034c0 2024-04-26T18:24:15.138 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:14.552+0000 7f8f99990200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:24:15.139 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:14.840+0000 7f8f99990200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:24:15.139 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:14.933+0000 7f8f99990200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:24:15.139 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:15.138+0000 7f8f99990200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:24:15.152 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:14.859+0000 7fe780cdf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:24:15.153 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:14.950+0000 7fe780cdf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:24:15.153 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:15.152+0000 7fe780cdf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:24:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.164+0000 7fb0dda77700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 msgr2=0x7fb0c40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.164+0000 7fb0dda77700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 0x7fb0c40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:24:15.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.226+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb0d81acd10 con 0x7fb0d81034c0 2024-04-26T18:24:15.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:15.045+0000 7fabde263200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:24:15.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:15.142+0000 7fabde263200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:24:15.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.226+0000 7fb0cf7fe700 1 -- 172.21.15.37:0/2696753721 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fb0d4013070 con 0x7fb0d81034c0 2024-04-26T18:24:15.227 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 msgr2=0x7fb0c40a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 0x7fb0c40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 msgr2=0x7fb0d810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40099c0 tx=0x7fb0d4004270 comp rx=0 tx=0).stop 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 shutdown_connections 2024-04-26T18:24:15.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7fb0c40a3d10 0x7fb0c40a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0d8109560 0x7fb0d810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0d81034c0 0x7fb0d810a340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 --2- 172.21.15.37:0/2696753721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0d8103e80 0x7fb0d810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 >> 172.21.15.37:0/2696753721 conn(0x7fb0d807a5d0 msgr2=0x7fb0d810e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 shutdown_connections 2024-04-26T18:24:15.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15.229+0000 7fb0e04dc700 1 -- 172.21.15.37:0/2696753721 wait complete. 2024-04-26T18:24:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:15 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2696753721' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:24:15.483 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:15.361+0000 7fabde263200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:24:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:15 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2696753721' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:24:15.503 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:15.344+0000 7fe780cdf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:15 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2696753721' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:24:15.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:15.332+0000 7f8f99990200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:15.822 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:15.571+0000 7fabde263200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:24:15.853 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T18:24:15.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T18:24:15.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T18:24:15.854 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:24:15.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:15.854 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:24:15.871 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42018sec preferred_lft 42018sec 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:24:15.872 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T18:24:15.872 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T18:24:15.873 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T18:24:15.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:16.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:16.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:16.065 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T18:24:16.092 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T18:24:16.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:16.095 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T18:24:16.167 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T18:24:16.167 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T18:24:16.167 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T18:24:16.167 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T18:24:16.167 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T18:24:16.168 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T18:24:16.252 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:15.936+0000 7fe780cdf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:24:16.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:16.325 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:15.931+0000 7f8f99990200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:24:16.368 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:16.229+0000 7fabde263200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:24:16.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:16.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:16.453 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T18:24:16.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:16.469 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-26T18:24:16.524 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:24:16.524 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T18:24:16.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T18:24:16.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T18:24:16.524 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T18:24:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:16.646 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T18:24:16.646 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T18:24:16.647 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T18:24:16.651 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:16.352+0000 7f8f99990200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:24:16.651 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:16.449+0000 7f8f99990200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:24:16.651 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:16.651+0000 7f8f99990200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:24:16.653 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:16.356+0000 7fe780cdf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:24:16.653 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:16.454+0000 7fe780cdf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:24:16.653 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:16.652+0000 7fe780cdf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:24:16.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:16.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:16.778 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:24:16.778 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T18:24:16.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-26T18:24:16.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T18:24:16.778 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T18:24:16.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:16.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:16.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:24:16.857 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:16.872 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:24:16.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:24:16.872 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:16.928 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T18:24:16.928 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T18:24:16.928 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/44bc6169-0263-4180-8dc4-a35378b8a772 2024-04-26T18:24:16.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:16.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/44bc6169-0263-4180-8dc4-a35378b8a772 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:24:16.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:16.695+0000 7fabde263200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:24:16.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:16.807+0000 7fabde263200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:24:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:24:17.027 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-26T18:24:17.028 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-26T18:24:17.134 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:24:17.134 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-26T18:24:17.134 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-26T18:24:17.134 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.120:3300,172.21.15.170:3300" 2024-04-26T18:24:17.134 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:3300,172.21.15.120:3300,172.21.15.170:3300:/volumes/qa/sv_1/44bc6169-0263-4180-8dc4-a35378b8a772 2024-04-26T18:24:17.135 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=755ef512-03f8-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T18:24:17.135 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T18:24:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T18:24:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T18:24:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T18:24:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T18:24:17.137 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T18:24:17.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:24:17.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:17.205 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:24:17.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:24:17.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:17.304 INFO:teuthology.orchestra.run.smithi037.stdout:client24848 2024-04-26T18:24:17.304 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:24:17.304 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:24:17.320 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:24:17.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:24:17.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:17.418 INFO:teuthology.orchestra.run.smithi037.stdout:client24848 2024-04-26T18:24:17.418 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:24:17.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:24:17.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:17.475 INFO:teuthology.orchestra.run.smithi037.stdout:client24848 2024-04-26T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/755ef512-03f8-11ef-bc93-c7b262605968.client24848/status') 2024-04-26T18:24:17.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:17.026+0000 7fabde263200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:24:17.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.172+0000 7fe780cdf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:24:17.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.263+0000 7fe780cdf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:24:17.511 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.177+0000 7f8f99990200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:24:17.511 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.269+0000 7f8f99990200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:24:17.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:24:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24848 (3)172.21.15.37:0/492035767 2024-04-26T18:24:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-26T18:24:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-26T18:24:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-26T18:24:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 2.6666e-05 s, 2.6 MB/s 2024-04-26T18:24:17.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:24:17.543 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:24:17.551 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:24:17.551 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:24:17.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- 172.21.15.37:0/1209739876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0102a80 msgr2=0x7f7ae0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/1209739876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0102a80 0x7f7ae0102f00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009900 tx=0x7f7acc02a410 comp rx=0 tx=0).stop 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- 172.21.15.37:0/1209739876 shutdown_connections 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/1209739876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0103440 0x7f7ae0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/1209739876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0102a80 0x7f7ae0102f00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/1209739876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0101890 0x7f7ae0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- 172.21.15.37:0/1209739876 >> 172.21.15.37:0/1209739876 conn(0x7f7ae00fce00 msgr2=0x7f7ae00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- 172.21.15.37:0/1209739876 shutdown_connections 2024-04-26T18:24:17.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- 172.21.15.37:0/1209739876 wait complete. 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 Processor -- start 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 -- start start 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.687+0000 7f7ae6aea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0102a80 0x7f7ae019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d3a0 con 0x7f7ae0103440 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d220 con 0x7f7ae0102a80 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010d520 con 0x7f7ae0101890 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae4886640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7adffff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0102a80 0x7f7ae019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:17.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae4886640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:36950/0 (socket says 172.21.15.37:36950) 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42046/0 (socket says 172.21.15.37:42046) 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae4886640 1 -- 172.21.15.37:0/2574156510 learned_addr learned my addr 172.21.15.37:0/2574156510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 msgr2=0x7f7ae006ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0102a80 msgr2=0x7f7ae019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0102a80 0x7f7ae019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 -- 172.21.15.37:0/2574156510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae01a29b0 con 0x7f7ae0103440 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae4886640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae5087640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e940 tx=0x7f7ad800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004d20 con 0x7f7ae0103440 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ad8005020 con 0x7f7ae0103440 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae01a2c40 con 0x7f7ae0103440 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.688+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80116d0 con 0x7f7ae0103440 2024-04-26T18:24:17.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.689+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae01a3110 con 0x7f7ae0103440 2024-04-26T18:24:17.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.690+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae0108c80 con 0x7f7ae0103440 2024-04-26T18:24:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.691+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7ad8005460 con 0x7f7ae0103440 2024-04-26T18:24:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.692+0000 7f7addffb640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 0x7f7ad00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.692+0000 7f7ae4886640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 msgr2=0x7f7ad00a6190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.692+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f7ad80d04b0 con 0x7f7ae0103440 2024-04-26T18:24:17.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.692+0000 7f7ae4886640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 0x7f7ad00a6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:24:17.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.697+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ad8016030 con 0x7f7ae0103440 2024-04-26T18:24:17.825 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.511+0000 7f8f99990200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:24:17.825 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.643+0000 7f8f99990200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:24:17.825 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.739+0000 7f8f99990200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:24:17.841 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.504+0000 7fe780cdf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:24:17.841 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.633+0000 7fe780cdf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:24:17.841 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.727+0000 7fe780cdf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:24:17.842 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:17.841+0000 7fe780cdf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.893+0000 7f7ae4886640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 msgr2=0x7f7ad00a6190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:17.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.893+0000 7f7ae4886640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 0x7f7ad00a6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:24:17.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.932+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7ae0063530 con 0x7f7ae0103440 2024-04-26T18:24:17.954 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:17.587+0000 7fabde263200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:24:17.955 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:17.690+0000 7fabde263200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:24:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.984+0000 7f7addffb640 1 -- 172.21.15.37:0/2574156510 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f7ad809c810 con 0x7f7ae0103440 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.986+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 msgr2=0x7f7ad00a6190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.986+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 0x7f7ad00a6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.986+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 msgr2=0x7f7ae01a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.986+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e940 tx=0x7f7ad800ee00 comp rx=0 tx=0).stop 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 shutdown_connections 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f7ad00a3cd0 0x7f7ad00a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae0102a80 0x7f7ae019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae0103440 0x7f7ae01a21c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 --2- 172.21.15.37:0/2574156510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ae0101890 0x7f7ae006ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 >> 172.21.15.37:0/2574156510 conn(0x7f7ae00fce00 msgr2=0x7f7ae0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 shutdown_connections 2024-04-26T18:24:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:17.987+0000 7f7ae6aea640 1 -- 172.21.15.37:0/2574156510 wait complete. 2024-04-26T18:24:17.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-26T18:24:18.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 -- 172.21.15.37:0/2267477297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103440 msgr2=0x7f189c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:18.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 --2- 172.21.15.37:0/2267477297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103440 0x7f189c109cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1894009ca0 tx=0x7f189402a8c0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 -- 172.21.15.37:0/2267477297 shutdown_connections 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 --2- 172.21.15.37:0/2267477297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f189c102a80 0x7f189c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 --2- 172.21.15.37:0/2267477297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103440 0x7f189c109cd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 --2- 172.21.15.37:0/2267477297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c101890 0x7f189c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 -- 172.21.15.37:0/2267477297 >> 172.21.15.37:0/2267477297 conn(0x7f189c0fce00 msgr2=0x7f189c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 -- 172.21.15.37:0/2267477297 shutdown_connections 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.106+0000 7f18a4093640 1 -- 172.21.15.37:0/2267477297 wait complete. 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 Processor -- start 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 -- start start 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 0x7f189c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f189c102a80 0x7f189c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c103440 0x7f189c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:18.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10d3c0 con 0x7f189c101890 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10d240 con 0x7f189c102a80 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a4093640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10d540 con 0x7f189c103440 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 0x7f189c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 0x7f189c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42064/0 (socket says 172.21.15.37:42064) 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 -- 172.21.15.37:0/2749575351 learned_addr learned my addr 172.21.15.37:0/2749575351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f189c102a80 0x7f189c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a2630640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c103440 0x7f189c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c103440 msgr2=0x7f189c1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c103440 0x7f189c1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f189c102a80 msgr2=0x7f189c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f189c102a80 0x7f189c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.107+0000 7f18a1e2f640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a0720 con 0x7f189c101890 2024-04-26T18:24:18.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f18a1e2f640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 0x7f189c198b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f188c00e9c0 tx=0x7f188c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:18.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c00cd80 con 0x7f189c101890 2024-04-26T18:24:18.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0035f0 con 0x7f189c101890 2024-04-26T18:24:18.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a0a10 con 0x7f189c101890 2024-04-26T18:24:18.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c010690 con 0x7f189c101890 2024-04-26T18:24:18.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.108+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a0f10 con 0x7f189c101890 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.109+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c1a11a0 con 0x7f189c101890 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.110+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f188c004000 con 0x7f189c101890 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.111+0000 7f1892ffd640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.111+0000 7f18a162e640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 msgr2=0x7f18880a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.111+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8277+0+0 (secure 0 0 0) 0x7f188c0d09e0 con 0x7f189c101890 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.111+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:24:18.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.115+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f188c09d330 con 0x7f189c101890 2024-04-26T18:24:18.144 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:17 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:24:18.161 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:24:18.162 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:17.854+0000 7f8f99990200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:18.206 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:17 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:24:18.206 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:17.954+0000 7fabde263200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:24:18.206 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:18.100+0000 7fabde263200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:24:18.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:18.205+0000 7fabde263200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:24:18.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.312+0000 7f18a162e640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 msgr2=0x7f18880a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:18.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.312+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:24:18.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.332+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f189c0fef90 con 0x7f18880a3ca0 2024-04-26T18:24:18.482 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:18.329+0000 7fabde263200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:24:18.502 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:18.143+0000 7fe780cdf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:24:18.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:18.161+0000 7f8f99990200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:24:18.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.713+0000 7f18a162e640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 msgr2=0x7f18880a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:18.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:18.713+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:24:18.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:18.643+0000 7fabde263200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:24:18.992 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:18.741+0000 7fe780cdf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:24:18.992 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y[33627]: 2024-04-26T18:24:18.955+0000 7fe780cdf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:24:18.992 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:24:18.992 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:24:18.992 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: Standby manager daemon y restarted 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: Standby manager daemon y started 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: Standby manager daemon z restarted 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: Standby manager daemon z started 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:24:18.993 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:18 smithi120 ceph-mon[32244]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:19.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.004+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f188c0992a0 con 0x7f189c101890 2024-04-26T18:24:19.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:18.761+0000 7f8f99990200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:24:19.075 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z[34552]: 2024-04-26T18:24:18.976+0000 7f8f99990200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: Standby manager daemon y restarted 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: Standby manager daemon y started 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: Standby manager daemon z restarted 2024-04-26T18:24:19.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: Standby manager daemon z started 2024-04-26T18:24:19.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:24:19.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:19.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:24:19.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:18 smithi170 ceph-mon[32697]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:19.284 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2574156510' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:24:19.284 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: Standby manager daemon y restarted 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: Standby manager daemon y started 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.120:0/240463880' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: Standby manager daemon z restarted 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: Standby manager daemon z started 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:24:19.285 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:18 smithi037 ceph-mon[29171]: from='mgr.? 172.21.15.170:0/1033813587' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:24:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.514+0000 7f18a162e640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 msgr2=0x7f18880a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/813673186 2024-04-26T18:24:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.514+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:24:19.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.528+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f188c0ce4d0 con 0x7f189c101890 2024-04-26T18:24:19.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.529+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 msgr2=0x7f18880a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:24:19.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:19.529+0000 7f1892ffd640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/813673186,v1:172.21.15.37:6801/813673186] conn(0x7f18880a3ca0 0x7f18880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.693 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.284+0000 7fabde263200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:24:19.693 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.516+0000 7fabde263200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.771+0000 7fab403c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.771+0000 7fab403c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.771+0000 7fab403c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.771+0000 7fab403c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.771+0000 7fab403c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.774+0000 7fab3c3bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.774+0000 7fab3c3bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.774+0000 7fab3c3bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.774+0000 7fab3c3bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:19.984 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:19.774+0000 7fab3c3bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: Active manager daemon x restarted 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: Activating manager daemon x 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: mgrmap e21: x(active, starting, since 0.0112987s), standbys: z, y 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:24:20.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:24:20.254 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:24:20.255 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: Manager daemon x is now available 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:24:20.256 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:20 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: Active manager daemon x restarted 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: Activating manager daemon x 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: mgrmap e21: x(active, starting, since 0.0112987s), standbys: z, y 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:24:20.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:20.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:24:20.328 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: Manager daemon x is now available 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:24:20.329 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:20 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:24:20.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: Active manager daemon x restarted 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: Activating manager daemon x 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: mgrmap e21: x(active, starting, since 0.0112987s), standbys: z, y 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:20.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:24:20.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: Manager daemon x is now available 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:24:20.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:24:20.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.532+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mon.0 v2:172.21.15.37:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f188c0797d0 con 0x7f189c101890 2024-04-26T18:24:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.533+0000 7f1892ffd640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1888035890 0x7f1888035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.533+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f189c0fef90 con 0x7f1888035890 2024-04-26T18:24:20.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.537+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1888035890 0x7f1888035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.539+0000 7f18a162e640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1888035890 0x7f1888035c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f189c19a010 tx=0x7f1898005f90 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.560+0000 7f1892ffd640 1 -- 172.21.15.37:0/2749575351 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f189c0fef90 con 0x7f1888035890 2024-04-26T18:24:20.560 INFO:teuthology.orchestra.run.smithi037.stdout:{} 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1888035890 msgr2=0x7f1888035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1888035890 0x7f1888035c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f189c19a010 tx=0x7f1898005f90 comp rx=0 tx=0).stop 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 msgr2=0x7f189c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 --2- 172.21.15.37:0/2749575351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c101890 0x7f189c198b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f188c00e9c0 tx=0x7f188c00ee80 comp rx=0 tx=0).stop 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a2630640 1 -- 172.21.15.37:0/2749575351 reap_dead start 2024-04-26T18:24:20.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 shutdown_connections 2024-04-26T18:24:20.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 >> 172.21.15.37:0/2749575351 conn(0x7f189c0fce00 msgr2=0x7f189c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 shutdown_connections 2024-04-26T18:24:20.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.562+0000 7f18a4093640 1 -- 172.21.15.37:0/2749575351 wait complete. 2024-04-26T18:24:20.574 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T18:24:20.591 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:24:20.606 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:24:20.606 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:24:20.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1495495671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1029a0 msgr2=0x7f4fec102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1495495671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1029a0 0x7f4fec102dc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe002a410 comp rx=0 tx=0).stop 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1495495671 shutdown_connections 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1495495671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec103b90 0x7f4fec104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1495495671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec104550 0x7f4fec10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1495495671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec1029a0 0x7f4fec102dc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1495495671 >> 172.21.15.37:0/1495495671 conn(0x7f4fec0fdf10 msgr2=0x7f4fec100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1495495671 shutdown_connections 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.712+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1495495671 wait complete. 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 Processor -- start 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 -- start start 2024-04-26T18:24:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec1029a0 0x7f4fec198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec104550 0x7f4fec1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10e4d0 con 0x7f4fec103b90 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10e350 con 0x7f4fec104550 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff30b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec10e650 con 0x7f4fec1029a0 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4febfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4febfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42192/0 (socket says 172.21.15.37:42192) 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4febfff640 1 -- 172.21.15.37:0/1704244055 learned_addr learned my addr 172.21.15.37:0/1704244055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff164d640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec104550 0x7f4fec1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.713+0000 7f4ff0e4c640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec1029a0 0x7f4fec198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 -- 172.21.15.37:0/1704244055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec1029a0 msgr2=0x7f4fec198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec1029a0 0x7f4fec198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 -- 172.21.15.37:0/1704244055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec104550 msgr2=0x7f4fec1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec104550 0x7f4fec1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 -- 172.21.15.37:0/1704244055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec1a0770 con 0x7f4fec103b90 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4febfff640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e9f0 tx=0x7f4fd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800cd80 con 0x7f4fec103b90 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd80108e0 con 0x7f4fec103b90 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec1a0a60 con 0x7f4fec103b90 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8018980 con 0x7f4fec103b90 2024-04-26T18:24:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.714+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a1000 con 0x7f4fec103b90 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.715+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec109cc0 con 0x7f4fec103b90 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.716+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4fd8010430 con 0x7f4fec103b90 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.717+0000 7f4fe9ffb640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 0x7f4fdc0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.717+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f4fd80d03d0 con 0x7f4fec103b90 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.717+0000 7f4ff0e4c640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 0x7f4fdc0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.718+0000 7f4ff0e4c640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 0x7f4fdc0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009230 tx=0x7f4fe00314a0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.722+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fd809c8e0 con 0x7f4fec103b90 2024-04-26T18:24:20.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:20.941+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4fec062d70 con 0x7f4fdc0a3ae0 2024-04-26T18:24:20.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:20.547+0000 7fab3a3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:20.982 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:20.547+0000 7fab3a3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:20.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:20.547+0000 7fab3a3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:20.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:20.547+0000 7fab3a3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:20.983 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:24:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:24:20.547+0000 7fab3a3b8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:21.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.007+0000 7f4fe9ffb640 1 -- 172.21.15.37:0/1704244055 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4fec062d70 con 0x7f4fdc0a3ae0 2024-04-26T18:24:21.008 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 msgr2=0x7f4fdc0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 0x7f4fdc0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009230 tx=0x7f4fe00314a0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 msgr2=0x7f4fec199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e9f0 tx=0x7f4fd800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 shutdown_connections 2024-04-26T18:24:21.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4fdc0a3ae0 0x7f4fdc0a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fec104550 0x7f4fec1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fec103b90 0x7f4fec199090 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 --2- 172.21.15.37:0/1704244055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fec1029a0 0x7f4fec198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 >> 172.21.15.37:0/1704244055 conn(0x7f4fec0fdf10 msgr2=0x7f4fec106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 shutdown_connections 2024-04-26T18:24:21.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.010+0000 7f4ff30b0640 1 -- 172.21.15.37:0/1704244055 wait complete. 2024-04-26T18:24:21.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T18:24:21.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: [26/Apr/2024:18:24:20] ENGINE Bus STARTING 2024-04-26T18:24:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: [26/Apr/2024:18:24:20] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T18:24:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: mgrmap e22: x(active, since 1.01778s), standbys: z, y 2024-04-26T18:24:21.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:21 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.140+0000 7f7e4cd24640 1 -- 172.21.15.37:0/3019005421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48105d40 msgr2=0x7f7e48108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.140+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/3019005421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48105d40 0x7f7e48108130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009900 tx=0x7f7e3c02a410 comp rx=0 tx=0).stop 2024-04-26T18:24:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 -- 172.21.15.37:0/3019005421 shutdown_connections 2024-04-26T18:24:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/3019005421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48108670 0x7f7e4810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/3019005421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48105d40 0x7f7e48108130 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/3019005421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48103410 0x7f7e48105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 -- 172.21.15.37:0/3019005421 >> 172.21.15.37:0/3019005421 conn(0x7f7e480fce00 msgr2=0x7f7e480ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 -- 172.21.15.37:0/3019005421 shutdown_connections 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 -- 172.21.15.37:0/3019005421 wait complete. 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 Processor -- start 2024-04-26T18:24:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.141+0000 7f7e4cd24640 1 -- start start 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48105d40 0x7f7e48199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48108670 0x7f7e481a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810e510 con 0x7f7e48103410 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810e390 con 0x7f7e48108670 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810e690 con 0x7f7e48105d40 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42206/0 (socket says 172.21.15.37:42206) 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 -- 172.21.15.37:0/664914635 learned_addr learned my addr 172.21.15.37:0/664914635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e45d9b640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48105d40 0x7f7e48199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e46d9d640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48108670 0x7f7e481a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 -- 172.21.15.37:0/664914635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48105d40 msgr2=0x7f7e48199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48105d40 0x7f7e48199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 -- 172.21.15.37:0/664914635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48108670 msgr2=0x7f7e481a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48108670 0x7f7e481a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 -- 172.21.15.37:0/664914635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481a0770 con 0x7f7e48103410 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4659c640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d950 tx=0x7f7e3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38017030 con 0x7f7e48103410 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.142+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e481a0a60 con 0x7f7e48103410 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.143+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e38004020 con 0x7f7e48103410 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.143+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380047a0 con 0x7f7e48103410 2024-04-26T18:24:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.143+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a0fd0 con 0x7f7e48103410 2024-04-26T18:24:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.144+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e480506c0 con 0x7f7e48103410 2024-04-26T18:24:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.145+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7e38009a30 con 0x7f7e48103410 2024-04-26T18:24:21.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.145+0000 7f7e377fe640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 0x7f7e300a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.145+0000 7f7e45d9b640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 0x7f7e300a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.145+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7e380cfb90 con 0x7f7e48103410 2024-04-26T18:24:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.146+0000 7f7e45d9b640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 0x7f7e300a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005ab0 tx=0x7f7e3c005a20 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.150+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e3809c730 con 0x7f7e48103410 2024-04-26T18:24:21.252 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: [26/Apr/2024:18:24:20] ENGINE Bus STARTING 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: [26/Apr/2024:18:24:20] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: mgrmap e22: x(active, since 1.01778s), standbys: z, y 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:21 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.365+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7e48062ed0 con 0x7f7e300a3ae0 2024-04-26T18:24:21.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: [26/Apr/2024:18:24:20] ENGINE Bus STARTING 2024-04-26T18:24:21.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: [26/Apr/2024:18:24:20] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T18:24:21.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: mgrmap e22: x(active, since 1.01778s), standbys: z, y 2024-04-26T18:24:21.409 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.409 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:21 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:21.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.426+0000 7f7e377fe640 1 -- 172.21.15.37:0/664914635 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7e48062ed0 con 0x7f7e300a3ae0 2024-04-26T18:24:21.427 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-04-26T18:24:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 msgr2=0x7f7e300a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 0x7f7e300a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005ab0 tx=0x7f7e3c005a20 comp rx=0 tx=0).stop 2024-04-26T18:24:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 msgr2=0x7f7e48198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d950 tx=0x7f7e3800de10 comp rx=0 tx=0).stop 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 shutdown_connections 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e300a3ae0 0x7f7e300a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e48108670 0x7f7e481a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e48105d40 0x7f7e48199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 --2- 172.21.15.37:0/664914635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e48103410 0x7f7e48198b50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 >> 172.21.15.37:0/664914635 conn(0x7f7e480fce00 msgr2=0x7f7e48103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 shutdown_connections 2024-04-26T18:24:21.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.430+0000 7f7e4cd24640 1 -- 172.21.15.37:0/664914635 wait complete. 2024-04-26T18:24:21.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T18:24:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 -- 172.21.15.37:0/317855146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 msgr2=0x7fb31c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 --2- 172.21.15.37:0/317855146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c101cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009900 tx=0x7fb30c02a410 comp rx=0 tx=0).stop 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 -- 172.21.15.37:0/317855146 shutdown_connections 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 --2- 172.21.15.37:0/317855146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 --2- 172.21.15.37:0/317855146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 0x7fb31c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 --2- 172.21.15.37:0/317855146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c101cb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 -- 172.21.15.37:0/317855146 >> 172.21.15.37:0/317855146 conn(0x7fb31c0fce20 msgr2=0x7fb31c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 -- 172.21.15.37:0/317855146 shutdown_connections 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.556+0000 7fb321932640 1 -- 172.21.15.37:0/317855146 wait complete. 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 Processor -- start 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 -- start start 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 0x7fb31c1043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c10d3e0 con 0x7fb31c101890 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c10d260 con 0x7fb31c102a80 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb321932640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c10d560 con 0x7fb31c103440 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42224/0 (socket says 172.21.15.37:42224) 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 0x7fb31c1043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 -- 172.21.15.37:0/3207199584 learned_addr learned my addr 172.21.15.37:0/3207199584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31a7fc640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 -- 172.21.15.37:0/3207199584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 msgr2=0x7fb31c1043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 0x7fb31c1043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 -- 172.21.15.37:0/3207199584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 msgr2=0x7fb31c103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.557+0000 7fb31affd640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c103e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb31affd640 1 -- 172.21.15.37:0/3207199584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c104c30 con 0x7fb31c101890 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb31a7fc640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb31affd640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb30c031500 tx=0x7fb30c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c03f070 con 0x7fb31c101890 2024-04-26T18:24:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb30c003e70 con 0x7fb31c101890 2024-04-26T18:24:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1a2b50 con 0x7fb31c101890 2024-04-26T18:24:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c035030 con 0x7fb31c101890 2024-04-26T18:24:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.558+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1a30a0 con 0x7fb31c101890 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.559+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c1a3270 con 0x7fb31c101890 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.560+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb30c042070 con 0x7fb31c101890 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.561+0000 7fb320930640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 0x7fb3040a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.561+0000 7fb31a7fc640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 0x7fb3040a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.561+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fb30c0f0df0 con 0x7fb31c101890 2024-04-26T18:24:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.561+0000 7fb31a7fc640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 0x7fb3040a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb31c105590 tx=0x7fb31000a5c0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.566+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb30c0bd950 con 0x7fb31c101890 2024-04-26T18:24:21.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.781+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb31c0feeb0 con 0x7fb3040a3a60 2024-04-26T18:24:21.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.788+0000 7fb320930640 1 -- 172.21.15.37:0/3207199584 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb31c0feeb0 con 0x7fb3040a3a60 2024-04-26T18:24:21.788 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:24:20", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 msgr2=0x7fb3040a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 0x7fb3040a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb31c105590 tx=0x7fb31000a5c0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 msgr2=0x7fb31c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb30c031500 tx=0x7fb30c003c80 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 shutdown_connections 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fb3040a3a60 0x7fb3040a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb31c102a80 0x7fb31c103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31c103440 0x7fb31c1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 --2- 172.21.15.37:0/3207199584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb31c101890 0x7fb31c105820 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.790+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 >> 172.21.15.37:0/3207199584 conn(0x7fb31c0fce20 msgr2=0x7fb31c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.791+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 shutdown_connections 2024-04-26T18:24:21.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.791+0000 7fb321932640 1 -- 172.21.15.37:0/3207199584 wait complete. 2024-04-26T18:24:21.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T18:24:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- 172.21.15.37:0/685602925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 msgr2=0x7f07b0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 --2- 172.21.15.37:0/685602925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b0108130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009900 tx=0x7f07a402a3f0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- 172.21.15.37:0/685602925 shutdown_connections 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 --2- 172.21.15.37:0/685602925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b0108130 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 --2- 172.21.15.37:0/685602925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0108670 0x7f07b010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 --2- 172.21.15.37:0/685602925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0103410 0x7f07b0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- 172.21.15.37:0/685602925 >> 172.21.15.37:0/685602925 conn(0x7f07b00fce00 msgr2=0x7f07b00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- 172.21.15.37:0/685602925 shutdown_connections 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- 172.21.15.37:0/685602925 wait complete. 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 Processor -- start 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.917+0000 7f07b667b640 1 -- start start 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0108670 0x7f07b0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07b010e520 con 0x7f07b0103410 2024-04-26T18:24:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f07b010e3a0 con 0x7f07b0105d40 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b667b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07b010e6a0 con 0x7f07b0108670 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42276/0 (socket says 172.21.15.37:42276) 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07af7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 -- 172.21.15.37:0/1521282506 learned_addr learned my addr 172.21.15.37:0/1521282506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07b4c18640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0108670 0x7f07b0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 -- 172.21.15.37:0/1521282506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0108670 msgr2=0x7f07b0101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0108670 0x7f07b0101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 -- 172.21.15.37:0/1521282506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 msgr2=0x7f07b01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 -- 172.21.15.37:0/1521282506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b0102190 con 0x7f07b0103410 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07af7fe640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.918+0000 7f07affff640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07a000e9f0 tx=0x7f07a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.919+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a000cd80 con 0x7f07b0103410 2024-04-26T18:24:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.919+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a00054d0 con 0x7f07b0103410 2024-04-26T18:24:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.919+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b01a2cb0 con 0x7f07b0103410 2024-04-26T18:24:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.919+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0010690 con 0x7f07b0103410 2024-04-26T18:24:21.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.919+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b01a3180 con 0x7f07b0103410 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.920+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b00506c0 con 0x7f07b0103410 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.921+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f07a0004e90 con 0x7f07b0103410 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.922+0000 7f07ad7fa640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 0x7f07980a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.922+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f07a00d0570 con 0x7f07b0103410 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.922+0000 7f07af7fe640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 0x7f07980a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:21.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.923+0000 7f07af7fe640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 0x7f07980a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07a4005ab0 tx=0x7f07a4005a20 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:21.927+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07a009d110 con 0x7f07b0103410 2024-04-26T18:24:22.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.023+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f07a0099080 con 0x7f07b0103410 2024-04-26T18:24:22.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.174+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f07b0062d70 con 0x7f07980a3a60 2024-04-26T18:24:22.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.181+0000 7f07ad7fa640 1 -- 172.21.15.37:0/1521282506 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f07b0062d70 con 0x7f07980a3a60 2024-04-26T18:24:22.182 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-26T18:24:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.183+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 msgr2=0x7f07980a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 0x7f07980a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07a4005ab0 tx=0x7f07a4005a20 comp rx=0 tx=0).stop 2024-04-26T18:24:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 msgr2=0x7f07b0102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07a000e9f0 tx=0x7f07a000edf0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 shutdown_connections 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f07980a3a60 0x7f07980a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07b0105d40 0x7f07b01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b0108670 0x7f07b0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 --2- 172.21.15.37:0/1521282506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b0103410 0x7f07b0102d80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 >> 172.21.15.37:0/1521282506 conn(0x7f07b00fce00 msgr2=0x7f07b0103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 shutdown_connections 2024-04-26T18:24:22.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:22.184+0000 7f07b667b640 1 -- 172.21.15.37:0/1521282506 wait complete. 2024-04-26T18:24:22.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: [26/Apr/2024:18:24:20] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: [26/Apr/2024:18:24:20] ENGINE Bus STARTED 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:22 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.196 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T18:24:22.225 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T18:24:22.236 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T18:24:22.254 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:24:22.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T18:24:22.268 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T18:24:22.281 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:24:22.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: [26/Apr/2024:18:24:20] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: [26/Apr/2024:18:24:20] ENGINE Bus STARTED 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:22 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: [26/Apr/2024:18:24:20] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: [26/Apr/2024:18:24:20] ENGINE Bus STARTED 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:22 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:22.552 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:23 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:23 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:23.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:23.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:23 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 -- 172.21.15.37:0/1079723618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc06bb50 msgr2=0x7fdccc06bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 --2- 172.21.15.37:0/1079723618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc06bb50 0x7fdccc06bfb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009990 tx=0x7fdcbc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 -- 172.21.15.37:0/1079723618 shutdown_connections 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 --2- 172.21.15.37:0/1079723618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 0x7fdccc1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 --2- 172.21.15.37:0/1079723618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc1094a0 0x7fdccc109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 --2- 172.21.15.37:0/1079723618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc06bb50 0x7fdccc06bfb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 -- 172.21.15.37:0/1079723618 >> 172.21.15.37:0/1079723618 conn(0x7fdccc0792d0 msgr2=0x7fdccc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 -- 172.21.15.37:0/1079723618 shutdown_connections 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.262+0000 7fdcd4334700 1 -- 172.21.15.37:0/1079723618 wait complete. 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 Processor -- start 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 -- start start 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc06bb50 0x7fdccc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 0x7fdccc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc1135b0 con 0x7fdccc1094a0 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc113430 con 0x7fdccc06c4f0 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd4334700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc113730 con 0x7fdccc06bb50 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42298/0 (socket says 172.21.15.37:42298) 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 -- 172.21.15.37:0/939755584 learned_addr learned my addr 172.21.15.37:0/939755584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd18cf700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 0x7fdccc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd20d0700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc06bb50 0x7fdccc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 -- 172.21.15.37:0/939755584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc06bb50 msgr2=0x7fdccc19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc06bb50 0x7fdccc19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 -- 172.21.15.37:0/939755584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 msgr2=0x7fdccc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.263+0000 7fdcd28d1700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 0x7fdccc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcd28d1700 1 -- 172.21.15.37:0/939755584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc19a0d0 con 0x7fdccc1094a0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcd28d1700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ee0 tx=0x7fdcc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400ddd0 con 0x7fdccc1094a0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc19a3c0 con 0x7fdccc1094a0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc4004020 con 0x7fdccc1094a0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4015660 con 0x7fdccc1094a0 2024-04-26T18:24:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.264+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc1a6680 con 0x7fdccc1094a0 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.265+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc06a110 con 0x7fdccc1094a0 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.266+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdcc40041d0 con 0x7fdccc1094a0 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.267+0000 7fdcc37fe700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 0x7fdcb80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.267+0000 7fdcd20d0700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 0x7fdcb80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.267+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fdcc40d05a0 con 0x7fdccc1094a0 2024-04-26T18:24:24.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.267+0000 7fdcd20d0700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 0x7fdcb80a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005fd0 tx=0x7fdcc8005d90 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.272+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdcc4010070 con 0x7fdccc1094a0 2024-04-26T18:24:24.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:24:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:24 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:24:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.487+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdccc063860 con 0x7fdcb80a3ed0 2024-04-26T18:24:24.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.495+0000 7fdcc37fe700 1 -- 172.21.15.37:0/939755584 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257753 (secure 0 0 0) 0x7fdccc063860 con 0x7fdcb80a3ed0 2024-04-26T18:24:24.499 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 msgr2=0x7fdcb80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 0x7fdcb80a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8005fd0 tx=0x7fdcc8005d90 comp rx=0 tx=0).stop 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 msgr2=0x7fdccc199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ee0 tx=0x7fdcc400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 shutdown_connections 2024-04-26T18:24:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc1094a0 0x7fdccc199970 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdccc06bb50 0x7fdccc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdcb80a3ed0 0x7fdcb80a6390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 --2- 172.21.15.37:0/939755584 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdccc06c4f0 0x7fdccc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.499+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 >> 172.21.15.37:0/939755584 conn(0x7fdccc0792d0 msgr2=0x7fdccc1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.500+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 shutdown_connections 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:24.500+0000 7fdcd4334700 1 -- 172.21.15.37:0/939755584 wait complete. 2024-04-26T18:24:24.502 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:24:24.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:24 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:24:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:24:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:24:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:24.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:24:24.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:24 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:24:25.312 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-26T18:24:23.539403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187401,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1304,"num_read_kb":812,"num_write":476,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182012,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385700,"kb_used_data":15708,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556124,"statfs":{"total":1151940427776,"available":1151545470976,"internally_reserved":0,"allocated":16084992,"data_stored":9495689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991830"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958989+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:24:19.958989+0000","last_peered":"2024-04-26T18:24:19.958989+0000","last_clean":"2024-04-26T18:24:19.958989+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:24:19.958989+0000","last_undegraded":"2024-04-26T18:24:19.958989+0000","last_fullsized":"2024-04-26T18:24:19.958989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779276+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:24:19.779276+0000","last_peered":"2024-04-26T18:24:19.779276+0000","last_clean":"2024-04-26T18:24:19.779276+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:24:19.779276+0000","last_undegraded":"2024-04-26T18:24:19.779276+0000","last_fullsized":"2024-04-26T18:24:19.779276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778888+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:24:19.778888+0000","last_peered":"2024-04-26T18:24:19.778888+0000","last_clean":"2024-04-26T18:24:19.778888+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:24:19.778888+0000","last_undegraded":"2024-04-26T18:24:19.778888+0000","last_fullsized":"2024-04-26T18:24:19.778888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525769+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:24:19.525769+0000","last_peered":"2024-04-26T18:24:19.525769+0000","last_clean":"2024-04-26T18:24:19.525769+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:24:19.525769+0000","last_undegraded":"2024-04-26T18:24:19.525769+0000","last_fullsized":"2024-04-26T18:24:19.525769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637144+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:24:19.637144+0000","last_peered":"2024-04-26T18:24:19.637144+0000","last_clean":"2024-04-26T18:24:19.637144+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:24:19.637144+0000","last_undegraded":"2024-04-26T18:24:19.637144+0000","last_fullsized":"2024-04-26T18:24:19.637144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778407+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:24:19.778407+0000","last_peered":"2024-04-26T18:24:19.778407+0000","last_clean":"2024-04-26T18:24:19.778407+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:24:19.778407+0000","last_undegraded":"2024-04-26T18:24:19.778407+0000","last_fullsized":"2024-04-26T18:24:19.778407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628784+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:24:19.628784+0000","last_peered":"2024-04-26T18:24:19.628784+0000","last_clean":"2024-04-26T18:24:19.628784+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:24:19.628784+0000","last_undegraded":"2024-04-26T18:24:19.628784+0000","last_fullsized":"2024-04-26T18:24:19.628784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.783243+0000","last_change":"2024-04-26T18:21:58.050914+0000","last_active":"2024-04-26T18:24:19.783243+0000","last_peered":"2024-04-26T18:24:19.783243+0000","last_clean":"2024-04-26T18:24:19.783243+0000","last_became_active":"2024-04-26T18:21:58.050400+0000","last_became_peered":"2024-04-26T18:21:58.050400+0000","last_unstale":"2024-04-26T18:24:19.783243+0000","last_undegraded":"2024-04-26T18:24:19.783243+0000","last_fullsized":"2024-04-26T18:24:19.783243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:37.789853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637426+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:24:19.637426+0000","last_peered":"2024-04-26T18:24:19.637426+0000","last_clean":"2024-04-26T18:24:19.637426+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:24:19.637426+0000","last_undegraded":"2024-04-26T18:24:19.637426+0000","last_fullsized":"2024-04-26T18:24:19.637426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778659+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:24:19.778659+0000","last_peered":"2024-04-26T18:24:19.778659+0000","last_clean":"2024-04-26T18:24:19.778659+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:24:19.778659+0000","last_undegraded":"2024-04-26T18:24:19.778659+0000","last_fullsized":"2024-04-26T18:24:19.778659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525634+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:24:19.525634+0000","last_peered":"2024-04-26T18:24:19.525634+0000","last_clean":"2024-04-26T18:24:19.525634+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:24:19.525634+0000","last_undegraded":"2024-04-26T18:24:19.525634+0000","last_fullsized":"2024-04-26T18:24:19.525634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524459+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:24:19.524459+0000","last_peered":"2024-04-26T18:24:19.524459+0000","last_clean":"2024-04-26T18:24:19.524459+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:24:19.524459+0000","last_undegraded":"2024-04-26T18:24:19.524459+0000","last_fullsized":"2024-04-26T18:24:19.524459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778429+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:24:19.778429+0000","last_peered":"2024-04-26T18:24:19.778429+0000","last_clean":"2024-04-26T18:24:19.778429+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:24:19.778429+0000","last_undegraded":"2024-04-26T18:24:19.778429+0000","last_fullsized":"2024-04-26T18:24:19.778429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778395+0000","last_change":"2024-04-26T18:21:58.051267+0000","last_active":"2024-04-26T18:24:19.778395+0000","last_peered":"2024-04-26T18:24:19.778395+0000","last_clean":"2024-04-26T18:24:19.778395+0000","last_became_active":"2024-04-26T18:21:58.050912+0000","last_became_peered":"2024-04-26T18:21:58.050912+0000","last_unstale":"2024-04-26T18:24:19.778395+0000","last_undegraded":"2024-04-26T18:24:19.778395+0000","last_fullsized":"2024-04-26T18:24:19.778395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:42.244892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782460+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:24:19.782460+0000","last_peered":"2024-04-26T18:24:19.782460+0000","last_clean":"2024-04-26T18:24:19.782460+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:24:19.782460+0000","last_undegraded":"2024-04-26T18:24:19.782460+0000","last_fullsized":"2024-04-26T18:24:19.782460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.959122+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:24:19.959122+0000","last_peered":"2024-04-26T18:24:19.959122+0000","last_clean":"2024-04-26T18:24:19.959122+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:24:19.959122+0000","last_undegraded":"2024-04-26T18:24:19.959122+0000","last_fullsized":"2024-04-26T18:24:19.959122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626731+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:24:19.626731+0000","last_peered":"2024-04-26T18:24:19.626731+0000","last_clean":"2024-04-26T18:24:19.626731+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:24:19.626731+0000","last_undegraded":"2024-04-26T18:24:19.626731+0000","last_fullsized":"2024-04-26T18:24:19.626731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780323+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:24:19.780323+0000","last_peered":"2024-04-26T18:24:19.780323+0000","last_clean":"2024-04-26T18:24:19.780323+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:24:19.780323+0000","last_undegraded":"2024-04-26T18:24:19.780323+0000","last_fullsized":"2024-04-26T18:24:19.780323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626753+0000","last_change":"2024-04-26T18:21:58.059181+0000","last_active":"2024-04-26T18:24:19.626753+0000","last_peered":"2024-04-26T18:24:19.626753+0000","last_clean":"2024-04-26T18:24:19.626753+0000","last_became_active":"2024-04-26T18:21:58.051741+0000","last_became_peered":"2024-04-26T18:21:58.051741+0000","last_unstale":"2024-04-26T18:24:19.626753+0000","last_undegraded":"2024-04-26T18:24:19.626753+0000","last_fullsized":"2024-04-26T18:24:19.626753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:55.251611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629163+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:24:19.629163+0000","last_peered":"2024-04-26T18:24:19.629163+0000","last_clean":"2024-04-26T18:24:19.629163+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:24:19.629163+0000","last_undegraded":"2024-04-26T18:24:19.629163+0000","last_fullsized":"2024-04-26T18:24:19.629163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630063+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:24:19.630063+0000","last_peered":"2024-04-26T18:24:19.630063+0000","last_clean":"2024-04-26T18:24:19.630063+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:24:19.630063+0000","last_undegraded":"2024-04-26T18:24:19.630063+0000","last_fullsized":"2024-04-26T18:24:19.630063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779767+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:24:19.779767+0000","last_peered":"2024-04-26T18:24:19.779767+0000","last_clean":"2024-04-26T18:24:19.779767+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:24:19.779767+0000","last_undegraded":"2024-04-26T18:24:19.779767+0000","last_fullsized":"2024-04-26T18:24:19.779767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629262+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:24:19.629262+0000","last_peered":"2024-04-26T18:24:19.629262+0000","last_clean":"2024-04-26T18:24:19.629262+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:24:19.629262+0000","last_undegraded":"2024-04-26T18:24:19.629262+0000","last_fullsized":"2024-04-26T18:24:19.629262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782545+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:24:19.782545+0000","last_peered":"2024-04-26T18:24:19.782545+0000","last_clean":"2024-04-26T18:24:19.782545+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:24:19.782545+0000","last_undegraded":"2024-04-26T18:24:19.782545+0000","last_fullsized":"2024-04-26T18:24:19.782545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629100+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:24:19.629100+0000","last_peered":"2024-04-26T18:24:19.629100+0000","last_clean":"2024-04-26T18:24:19.629100+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:24:19.629100+0000","last_undegraded":"2024-04-26T18:24:19.629100+0000","last_fullsized":"2024-04-26T18:24:19.629100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525994+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:24:19.525994+0000","last_peered":"2024-04-26T18:24:19.525994+0000","last_clean":"2024-04-26T18:24:19.525994+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:24:19.525994+0000","last_undegraded":"2024-04-26T18:24:19.525994+0000","last_fullsized":"2024-04-26T18:24:19.525994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":220,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:20.219716+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:24:20.219716+0000","last_peered":"2024-04-26T18:24:20.219716+0000","last_clean":"2024-04-26T18:24:20.219716+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:24:20.219716+0000","last_undegraded":"2024-04-26T18:24:20.219716+0000","last_fullsized":"2024-04-26T18:24:20.219716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628856+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:24:19.628856+0000","last_peered":"2024-04-26T18:24:19.628856+0000","last_clean":"2024-04-26T18:24:19.628856+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:24:19.628856+0000","last_undegraded":"2024-04-26T18:24:19.628856+0000","last_fullsized":"2024-04-26T18:24:19.628856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637053+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:24:19.637053+0000","last_peered":"2024-04-26T18:24:19.637053+0000","last_clean":"2024-04-26T18:24:19.637053+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:24:19.637053+0000","last_undegraded":"2024-04-26T18:24:19.637053+0000","last_fullsized":"2024-04-26T18:24:19.637053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526234+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:24:19.526234+0000","last_peered":"2024-04-26T18:24:19.526234+0000","last_clean":"2024-04-26T18:24:19.526234+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:24:19.526234+0000","last_undegraded":"2024-04-26T18:24:19.526234+0000","last_fullsized":"2024-04-26T18:24:19.526234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524315+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:24:19.524315+0000","last_peered":"2024-04-26T18:24:19.524315+0000","last_clean":"2024-04-26T18:24:19.524315+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:24:19.524315+0000","last_undegraded":"2024-04-26T18:24:19.524315+0000","last_fullsized":"2024-04-26T18:24:19.524315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628948+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:24:19.628948+0000","last_peered":"2024-04-26T18:24:19.628948+0000","last_clean":"2024-04-26T18:24:19.628948+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:24:19.628948+0000","last_undegraded":"2024-04-26T18:24:19.628948+0000","last_fullsized":"2024-04-26T18:24:19.628948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525775+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:24:19.525775+0000","last_peered":"2024-04-26T18:24:19.525775+0000","last_clean":"2024-04-26T18:24:19.525775+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:24:19.525775+0000","last_undegraded":"2024-04-26T18:24:19.525775+0000","last_fullsized":"2024-04-26T18:24:19.525775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626253+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:24:19.626253+0000","last_peered":"2024-04-26T18:24:19.626253+0000","last_clean":"2024-04-26T18:24:19.626253+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:24:19.626253+0000","last_undegraded":"2024-04-26T18:24:19.626253+0000","last_fullsized":"2024-04-26T18:24:19.626253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780229+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:24:19.780229+0000","last_peered":"2024-04-26T18:24:19.780229+0000","last_clean":"2024-04-26T18:24:19.780229+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:24:19.780229+0000","last_undegraded":"2024-04-26T18:24:19.780229+0000","last_fullsized":"2024-04-26T18:24:19.780229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782966+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:24:19.782966+0000","last_peered":"2024-04-26T18:24:19.782966+0000","last_clean":"2024-04-26T18:24:19.782966+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:24:19.782966+0000","last_undegraded":"2024-04-26T18:24:19.782966+0000","last_fullsized":"2024-04-26T18:24:19.782966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778729+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:24:19.778729+0000","last_peered":"2024-04-26T18:24:19.778729+0000","last_clean":"2024-04-26T18:24:19.778729+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:24:19.778729+0000","last_undegraded":"2024-04-26T18:24:19.778729+0000","last_fullsized":"2024-04-26T18:24:19.778729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779611+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:24:19.779611+0000","last_peered":"2024-04-26T18:24:19.779611+0000","last_clean":"2024-04-26T18:24:19.779611+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:24:19.779611+0000","last_undegraded":"2024-04-26T18:24:19.779611+0000","last_fullsized":"2024-04-26T18:24:19.779611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526045+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:24:19.526045+0000","last_peered":"2024-04-26T18:24:19.526045+0000","last_clean":"2024-04-26T18:24:19.526045+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:24:19.526045+0000","last_undegraded":"2024-04-26T18:24:19.526045+0000","last_fullsized":"2024-04-26T18:24:19.526045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782568+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:24:19.782568+0000","last_peered":"2024-04-26T18:24:19.782568+0000","last_clean":"2024-04-26T18:24:19.782568+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:24:19.782568+0000","last_undegraded":"2024-04-26T18:24:19.782568+0000","last_fullsized":"2024-04-26T18:24:19.782568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":428,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:20.219700+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:24:20.219700+0000","last_peered":"2024-04-26T18:24:20.219700+0000","last_clean":"2024-04-26T18:24:20.219700+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:24:20.219700+0000","last_undegraded":"2024-04-26T18:24:20.219700+0000","last_fullsized":"2024-04-26T18:24:20.219700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":246,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629095+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:24:19.629095+0000","last_peered":"2024-04-26T18:24:19.629095+0000","last_clean":"2024-04-26T18:24:19.629095+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:24:19.629095+0000","last_undegraded":"2024-04-26T18:24:19.629095+0000","last_fullsized":"2024-04-26T18:24:19.629095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780221+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:24:19.780221+0000","last_peered":"2024-04-26T18:24:19.780221+0000","last_clean":"2024-04-26T18:24:19.780221+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:24:19.780221+0000","last_undegraded":"2024-04-26T18:24:19.780221+0000","last_fullsized":"2024-04-26T18:24:19.780221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524345+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:24:19.524345+0000","last_peered":"2024-04-26T18:24:19.524345+0000","last_clean":"2024-04-26T18:24:19.524345+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:24:19.524345+0000","last_undegraded":"2024-04-26T18:24:19.524345+0000","last_fullsized":"2024-04-26T18:24:19.524345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525928+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:24:19.525928+0000","last_peered":"2024-04-26T18:24:19.525928+0000","last_clean":"2024-04-26T18:24:19.525928+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:24:19.525928+0000","last_undegraded":"2024-04-26T18:24:19.525928+0000","last_fullsized":"2024-04-26T18:24:19.525928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778618+0000","last_change":"2024-04-26T18:21:58.049111+0000","last_active":"2024-04-26T18:24:19.778618+0000","last_peered":"2024-04-26T18:24:19.778618+0000","last_clean":"2024-04-26T18:24:19.778618+0000","last_became_active":"2024-04-26T18:21:58.048723+0000","last_became_peered":"2024-04-26T18:21:58.048723+0000","last_unstale":"2024-04-26T18:24:19.778618+0000","last_undegraded":"2024-04-26T18:24:19.778618+0000","last_fullsized":"2024-04-26T18:24:19.778618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:26.107528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'4","reported_seq":214,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:20.467901+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:24:20.467901+0000","last_peered":"2024-04-26T18:24:20.467901+0000","last_clean":"2024-04-26T18:24:20.467901+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:24:20.467901+0000","last_undegraded":"2024-04-26T18:24:20.467901+0000","last_fullsized":"2024-04-26T18:24:20.467901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778281+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:24:19.778281+0000","last_peered":"2024-04-26T18:24:19.778281+0000","last_clean":"2024-04-26T18:24:19.778281+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:24:19.778281+0000","last_undegraded":"2024-04-26T18:24:19.778281+0000","last_fullsized":"2024-04-26T18:24:19.778281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958688+0000","last_change":"2024-04-26T18:22:59.027035+0000","last_active":"2024-04-26T18:24:19.958688+0000","last_peered":"2024-04-26T18:24:19.958688+0000","last_clean":"2024-04-26T18:24:19.958688+0000","last_became_active":"2024-04-26T18:22:59.015160+0000","last_became_peered":"2024-04-26T18:22:59.015160+0000","last_unstale":"2024-04-26T18:24:19.958688+0000","last_undegraded":"2024-04-26T18:24:19.958688+0000","last_fullsized":"2024-04-26T18:24:19.958688+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:04.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626360+0000","last_change":"2024-04-26T18:21:58.052169+0000","last_active":"2024-04-26T18:24:19.626360+0000","last_peered":"2024-04-26T18:24:19.626360+0000","last_clean":"2024-04-26T18:24:19.626360+0000","last_became_active":"2024-04-26T18:21:58.051756+0000","last_became_peered":"2024-04-26T18:21:58.051756+0000","last_unstale":"2024-04-26T18:24:19.626360+0000","last_undegraded":"2024-04-26T18:24:19.626360+0000","last_fullsized":"2024-04-26T18:24:19.626360+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:11.730323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629270+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:24:19.629270+0000","last_peered":"2024-04-26T18:24:19.629270+0000","last_clean":"2024-04-26T18:24:19.629270+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:24:19.629270+0000","last_undegraded":"2024-04-26T18:24:19.629270+0000","last_fullsized":"2024-04-26T18:24:19.629270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958915+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:24:19.958915+0000","last_peered":"2024-04-26T18:24:19.958915+0000","last_clean":"2024-04-26T18:24:19.958915+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:24:19.958915+0000","last_undegraded":"2024-04-26T18:24:19.958915+0000","last_fullsized":"2024-04-26T18:24:19.958915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626667+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:24:19.626667+0000","last_peered":"2024-04-26T18:24:19.626667+0000","last_clean":"2024-04-26T18:24:19.626667+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:24:19.626667+0000","last_undegraded":"2024-04-26T18:24:19.626667+0000","last_fullsized":"2024-04-26T18:24:19.626667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637345+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:24:19.637345+0000","last_peered":"2024-04-26T18:24:19.637345+0000","last_clean":"2024-04-26T18:24:19.637345+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:24:19.637345+0000","last_undegraded":"2024-04-26T18:24:19.637345+0000","last_fullsized":"2024-04-26T18:24:19.637345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778652+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:24:19.778652+0000","last_peered":"2024-04-26T18:24:19.778652+0000","last_clean":"2024-04-26T18:24:19.778652+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:24:19.778652+0000","last_undegraded":"2024-04-26T18:24:19.778652+0000","last_fullsized":"2024-04-26T18:24:19.778652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637164+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:24:19.637164+0000","last_peered":"2024-04-26T18:24:19.637164+0000","last_clean":"2024-04-26T18:24:19.637164+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:24:19.637164+0000","last_undegraded":"2024-04-26T18:24:19.637164+0000","last_fullsized":"2024-04-26T18:24:19.637164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526068+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:24:19.526068+0000","last_peered":"2024-04-26T18:24:19.526068+0000","last_clean":"2024-04-26T18:24:19.526068+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:24:19.526068+0000","last_undegraded":"2024-04-26T18:24:19.526068+0000","last_fullsized":"2024-04-26T18:24:19.526068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779019+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:24:19.779019+0000","last_peered":"2024-04-26T18:24:19.779019+0000","last_clean":"2024-04-26T18:24:19.779019+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:24:19.779019+0000","last_undegraded":"2024-04-26T18:24:19.779019+0000","last_fullsized":"2024-04-26T18:24:19.779019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629371+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:24:19.629371+0000","last_peered":"2024-04-26T18:24:19.629371+0000","last_clean":"2024-04-26T18:24:19.629371+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:24:19.629371+0000","last_undegraded":"2024-04-26T18:24:19.629371+0000","last_fullsized":"2024-04-26T18:24:19.629371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525607+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:24:19.525607+0000","last_peered":"2024-04-26T18:24:19.525607+0000","last_clean":"2024-04-26T18:24:19.525607+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:24:19.525607+0000","last_undegraded":"2024-04-26T18:24:19.525607+0000","last_fullsized":"2024-04-26T18:24:19.525607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778392+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:24:19.778392+0000","last_peered":"2024-04-26T18:24:19.778392+0000","last_clean":"2024-04-26T18:24:19.778392+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:24:19.778392+0000","last_undegraded":"2024-04-26T18:24:19.778392+0000","last_fullsized":"2024-04-26T18:24:19.778392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524170+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:24:19.524170+0000","last_peered":"2024-04-26T18:24:19.524170+0000","last_clean":"2024-04-26T18:24:19.524170+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:24:19.524170+0000","last_undegraded":"2024-04-26T18:24:19.524170+0000","last_fullsized":"2024-04-26T18:24:19.524170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778505+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:24:19.778505+0000","last_peered":"2024-04-26T18:24:19.778505+0000","last_clean":"2024-04-26T18:24:19.778505+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:24:19.778505+0000","last_undegraded":"2024-04-26T18:24:19.778505+0000","last_fullsized":"2024-04-26T18:24:19.778505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629160+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:24:19.629160+0000","last_peered":"2024-04-26T18:24:19.629160+0000","last_clean":"2024-04-26T18:24:19.629160+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:24:19.629160+0000","last_undegraded":"2024-04-26T18:24:19.629160+0000","last_fullsized":"2024-04-26T18:24:19.629160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524566+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:24:19.524566+0000","last_peered":"2024-04-26T18:24:19.524566+0000","last_clean":"2024-04-26T18:24:19.524566+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:24:19.524566+0000","last_undegraded":"2024-04-26T18:24:19.524566+0000","last_fullsized":"2024-04-26T18:24:19.524566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.740682+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:24:19.740682+0000","last_peered":"2024-04-26T18:24:19.740682+0000","last_clean":"2024-04-26T18:24:19.740682+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:24:19.740682+0000","last_undegraded":"2024-04-26T18:24:19.740682+0000","last_fullsized":"2024-04-26T18:24:19.740682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630066+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:24:19.630066+0000","last_peered":"2024-04-26T18:24:19.630066+0000","last_clean":"2024-04-26T18:24:19.630066+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:24:19.630066+0000","last_undegraded":"2024-04-26T18:24:19.630066+0000","last_fullsized":"2024-04-26T18:24:19.630066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525682+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:24:19.525682+0000","last_peered":"2024-04-26T18:24:19.525682+0000","last_clean":"2024-04-26T18:24:19.525682+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:24:19.525682+0000","last_undegraded":"2024-04-26T18:24:19.525682+0000","last_fullsized":"2024-04-26T18:24:19.525682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779408+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:24:19.779408+0000","last_peered":"2024-04-26T18:24:19.779408+0000","last_clean":"2024-04-26T18:24:19.779408+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:24:19.779408+0000","last_undegraded":"2024-04-26T18:24:19.779408+0000","last_fullsized":"2024-04-26T18:24:19.779408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782477+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:24:19.782477+0000","last_peered":"2024-04-26T18:24:19.782477+0000","last_clean":"2024-04-26T18:24:19.782477+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:24:19.782477+0000","last_undegraded":"2024-04-26T18:24:19.782477+0000","last_fullsized":"2024-04-26T18:24:19.782477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628827+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:24:19.628827+0000","last_peered":"2024-04-26T18:24:19.628827+0000","last_clean":"2024-04-26T18:24:19.628827+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:24:19.628827+0000","last_undegraded":"2024-04-26T18:24:19.628827+0000","last_fullsized":"2024-04-26T18:24:19.628827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628902+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:24:19.628902+0000","last_peered":"2024-04-26T18:24:19.628902+0000","last_clean":"2024-04-26T18:24:19.628902+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:24:19.628902+0000","last_undegraded":"2024-04-26T18:24:19.628902+0000","last_fullsized":"2024-04-26T18:24:19.628902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958753+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:24:19.958753+0000","last_peered":"2024-04-26T18:24:19.958753+0000","last_clean":"2024-04-26T18:24:19.958753+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:24:19.958753+0000","last_undegraded":"2024-04-26T18:24:19.958753+0000","last_fullsized":"2024-04-26T18:24:19.958753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958835+0000","last_change":"2024-04-26T18:21:58.057410+0000","last_active":"2024-04-26T18:24:19.958835+0000","last_peered":"2024-04-26T18:24:19.958835+0000","last_clean":"2024-04-26T18:24:19.958835+0000","last_became_active":"2024-04-26T18:21:58.054453+0000","last_became_peered":"2024-04-26T18:21:58.054453+0000","last_unstale":"2024-04-26T18:24:19.958835+0000","last_undegraded":"2024-04-26T18:24:19.958835+0000","last_fullsized":"2024-04-26T18:24:19.958835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:22.856834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629599+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:24:19.629599+0000","last_peered":"2024-04-26T18:24:19.629599+0000","last_clean":"2024-04-26T18:24:19.629599+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:24:19.629599+0000","last_undegraded":"2024-04-26T18:24:19.629599+0000","last_fullsized":"2024-04-26T18:24:19.629599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778674+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:24:19.778674+0000","last_peered":"2024-04-26T18:24:19.778674+0000","last_clean":"2024-04-26T18:24:19.778674+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:24:19.778674+0000","last_undegraded":"2024-04-26T18:24:19.778674+0000","last_fullsized":"2024-04-26T18:24:19.778674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525697+0000","last_change":"2024-04-26T18:21:58.069561+0000","last_active":"2024-04-26T18:24:19.525697+0000","last_peered":"2024-04-26T18:24:19.525697+0000","last_clean":"2024-04-26T18:24:19.525697+0000","last_became_active":"2024-04-26T18:21:58.053199+0000","last_became_peered":"2024-04-26T18:21:58.053199+0000","last_unstale":"2024-04-26T18:24:19.525697+0000","last_undegraded":"2024-04-26T18:24:19.525697+0000","last_fullsized":"2024-04-26T18:24:19.525697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:27.166045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778751+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:24:19.778751+0000","last_peered":"2024-04-26T18:24:19.778751+0000","last_clean":"2024-04-26T18:24:19.778751+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:24:19.778751+0000","last_undegraded":"2024-04-26T18:24:19.778751+0000","last_fullsized":"2024-04-26T18:24:19.778751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":80,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.738292+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:24:19.738292+0000","last_peered":"2024-04-26T18:24:19.738292+0000","last_clean":"2024-04-26T18:24:19.738292+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:24:19.738292+0000","last_undegraded":"2024-04-26T18:24:19.738292+0000","last_fullsized":"2024-04-26T18:24:19.738292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778454+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:24:19.778454+0000","last_peered":"2024-04-26T18:24:19.778454+0000","last_clean":"2024-04-26T18:24:19.778454+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:24:19.778454+0000","last_undegraded":"2024-04-26T18:24:19.778454+0000","last_fullsized":"2024-04-26T18:24:19.778454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524399+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:24:19.524399+0000","last_peered":"2024-04-26T18:24:19.524399+0000","last_clean":"2024-04-26T18:24:19.524399+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:24:19.524399+0000","last_undegraded":"2024-04-26T18:24:19.524399+0000","last_fullsized":"2024-04-26T18:24:19.524399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778847+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:24:19.778847+0000","last_peered":"2024-04-26T18:24:19.778847+0000","last_clean":"2024-04-26T18:24:19.778847+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:24:19.778847+0000","last_undegraded":"2024-04-26T18:24:19.778847+0000","last_fullsized":"2024-04-26T18:24:19.778847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958882+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:24:19.958882+0000","last_peered":"2024-04-26T18:24:19.958882+0000","last_clean":"2024-04-26T18:24:19.958882+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:24:19.958882+0000","last_undegraded":"2024-04-26T18:24:19.958882+0000","last_fullsized":"2024-04-26T18:24:19.958882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780084+0000","last_change":"2024-04-26T18:21:58.052815+0000","last_active":"2024-04-26T18:24:19.780084+0000","last_peered":"2024-04-26T18:24:19.780084+0000","last_clean":"2024-04-26T18:24:19.780084+0000","last_became_active":"2024-04-26T18:21:58.052405+0000","last_became_peered":"2024-04-26T18:21:58.052405+0000","last_unstale":"2024-04-26T18:24:19.780084+0000","last_undegraded":"2024-04-26T18:24:19.780084+0000","last_fullsized":"2024-04-26T18:24:19.780084+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:29.289089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780025+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:24:19.780025+0000","last_peered":"2024-04-26T18:24:19.780025+0000","last_clean":"2024-04-26T18:24:19.780025+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:24:19.780025+0000","last_undegraded":"2024-04-26T18:24:19.780025+0000","last_fullsized":"2024-04-26T18:24:19.780025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778513+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:24:19.778513+0000","last_peered":"2024-04-26T18:24:19.778513+0000","last_clean":"2024-04-26T18:24:19.778513+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:24:19.778513+0000","last_undegraded":"2024-04-26T18:24:19.778513+0000","last_fullsized":"2024-04-26T18:24:19.778513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524720+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:24:19.524720+0000","last_peered":"2024-04-26T18:24:19.524720+0000","last_clean":"2024-04-26T18:24:19.524720+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:24:19.524720+0000","last_undegraded":"2024-04-26T18:24:19.524720+0000","last_fullsized":"2024-04-26T18:24:19.524720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958902+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:24:19.958902+0000","last_peered":"2024-04-26T18:24:19.958902+0000","last_clean":"2024-04-26T18:24:19.958902+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:24:19.958902+0000","last_undegraded":"2024-04-26T18:24:19.958902+0000","last_fullsized":"2024-04-26T18:24:19.958902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780397+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:24:19.780397+0000","last_peered":"2024-04-26T18:24:19.780397+0000","last_clean":"2024-04-26T18:24:19.780397+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:24:19.780397+0000","last_undegraded":"2024-04-26T18:24:19.780397+0000","last_fullsized":"2024-04-26T18:24:19.780397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630028+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:24:19.630028+0000","last_peered":"2024-04-26T18:24:19.630028+0000","last_clean":"2024-04-26T18:24:19.630028+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:24:19.630028+0000","last_undegraded":"2024-04-26T18:24:19.630028+0000","last_fullsized":"2024-04-26T18:24:19.630028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525817+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:24:19.525817+0000","last_peered":"2024-04-26T18:24:19.525817+0000","last_clean":"2024-04-26T18:24:19.525817+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:24:19.525817+0000","last_undegraded":"2024-04-26T18:24:19.525817+0000","last_fullsized":"2024-04-26T18:24:19.525817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626199+0000","last_change":"2024-04-26T18:21:58.051597+0000","last_active":"2024-04-26T18:24:19.626199+0000","last_peered":"2024-04-26T18:24:19.626199+0000","last_clean":"2024-04-26T18:24:19.626199+0000","last_became_active":"2024-04-26T18:21:58.051139+0000","last_became_peered":"2024-04-26T18:21:58.051139+0000","last_unstale":"2024-04-26T18:24:19.626199+0000","last_undegraded":"2024-04-26T18:24:19.626199+0000","last_fullsized":"2024-04-26T18:24:19.626199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.289734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782686+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:24:19.782686+0000","last_peered":"2024-04-26T18:24:19.782686+0000","last_clean":"2024-04-26T18:24:19.782686+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:24:19.782686+0000","last_undegraded":"2024-04-26T18:24:19.782686+0000","last_fullsized":"2024-04-26T18:24:19.782686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524626+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:24:19.524626+0000","last_peered":"2024-04-26T18:24:19.524626+0000","last_clean":"2024-04-26T18:24:19.524626+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:24:19.524626+0000","last_undegraded":"2024-04-26T18:24:19.524626+0000","last_fullsized":"2024-04-26T18:24:19.524626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958776+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:24:19.958776+0000","last_peered":"2024-04-26T18:24:19.958776+0000","last_clean":"2024-04-26T18:24:19.958776+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:24:19.958776+0000","last_undegraded":"2024-04-26T18:24:19.958776+0000","last_fullsized":"2024-04-26T18:24:19.958776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596799,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1168,"num_read_kb":693,"num_write":293,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1276,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1790397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40520,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704632,"statfs":{"total":95995035648,"available":95953543168,"internally_reserved":0,"allocated":1044480,"data_stored":506183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28848,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716304,"statfs":{"total":95995035648,"available":95965495296,"internally_reserved":0,"allocated":1875968,"data_stored":1296602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1445888,"data_stored":899523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1433600,"data_stored":898749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1099080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":505696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":573130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":506967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40596,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704556,"statfs":{"total":95995035648,"available":95953465344,"internally_reserved":0,"allocated":1122304,"data_stored":574127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1030823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920857,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":506827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1097982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:24:25.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:24:25.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:25 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:25 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-26T18:24:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:24:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:25 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:25.576 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:26.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:26.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:26.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:24:26.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi120:/etc/ceph/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: Updating smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:24:26.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:26.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:24:27.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 -- 172.21.15.37:0/1528567808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 msgr2=0x7f02581098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:27.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 --2- 172.21.15.37:0/1528567808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f02581098b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0248009990 tx=0x7f0248009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 -- 172.21.15.37:0/1528567808 shutdown_connections 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 --2- 172.21.15.37:0/1528567808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 0x7f0258110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 --2- 172.21.15.37:0/1528567808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f02581098b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 --2- 172.21.15.37:0/1528567808 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 0x7f025806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 -- 172.21.15.37:0/1528567808 >> 172.21.15.37:0/1528567808 conn(0x7f02580793c0 msgr2=0x7f02580797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 -- 172.21.15.37:0/1528567808 shutdown_connections 2024-04-26T18:24:27.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 -- 172.21.15.37:0/1528567808 wait complete. 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.157+0000 7f025dab7700 1 Processor -- start 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 -- start start 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 0x7f025819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 0x7f025819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f02581135e0 con 0x7f02581094d0 2024-04-26T18:24:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0258113460 con 0x7f025806bbd0 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f025dab7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0258113760 con 0x7f025806c590 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53598/0 (socket says 172.21.15.37:53598) 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 -- 172.21.15.37:0/3231409847 learned_addr learned my addr 172.21.15.37:0/3231409847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f02577fe700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 0x7f025819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0256ffd700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 0x7f025819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 -- 172.21.15.37:0/3231409847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 msgr2=0x7f025819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 0x7f025819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 -- 172.21.15.37:0/3231409847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 msgr2=0x7f025819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 0x7f025819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.158+0000 7f0257fff700 1 -- 172.21.15.37:0/3231409847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025819a0a0 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f0257fff700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f025000ada0 tx=0x7f025000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250017020 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025819a390 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f025000ce30 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02500054a0 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.159+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a6650 con 0x7f02581094d0 2024-04-26T18:24:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.160+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025810ba40 con 0x7f02581094d0 2024-04-26T18:24:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.161+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f025000c980 con 0x7f02581094d0 2024-04-26T18:24:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.162+0000 7f0254ff9700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 0x7f02400a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.162+0000 7f02577fe700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 0x7f02400a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.162+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f02500d02c0 con 0x7f02581094d0 2024-04-26T18:24:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.162+0000 7f02577fe700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 0x7f02400a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0248006010 tx=0x7f024800b540 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:27.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.167+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0250021020 con 0x7f02581094d0 2024-04-26T18:24:27.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:27 smithi037 ceph-mon[29171]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-04-26T18:24:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.371+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f025819ae70 con 0x7f02400a3ed0 2024-04-26T18:24:27.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.379+0000 7f0254ff9700 1 -- 172.21.15.37:0/3231409847 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257769 (secure 0 0 0) 0x7f025819ae70 con 0x7f02400a3ed0 2024-04-26T18:24:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:27.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 msgr2=0x7f02400a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:27.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 0x7f02400a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0248006010 tx=0x7f024800b540 comp rx=0 tx=0).stop 2024-04-26T18:24:27.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 msgr2=0x7f0258199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f025000ada0 tx=0x7f025000c330 comp rx=0 tx=0).stop 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 shutdown_connections 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02581094d0 0x7f0258199940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f02400a3ed0 0x7f02400a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f025806bbd0 0x7f025819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 --2- 172.21.15.37:0/3231409847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f025806c590 0x7f025819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.381+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 >> 172.21.15.37:0/3231409847 conn(0x7f02580793c0 msgr2=0x7f0258100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.382+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 shutdown_connections 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:27.382+0000 7f025dab7700 1 -- 172.21.15.37:0/3231409847 wait complete. 2024-04-26T18:24:27.385 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:24:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:27 smithi120 ceph-mon[32244]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-04-26T18:24:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:27 smithi170 ceph-mon[32697]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 90 KiB/s wr, 91 op/s 2024-04-26T18:24:28.073 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-26T18:24:25.539715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187401,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1640,"num_read_kb":1230,"num_write":597,"num_write_kb":4826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182012,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":932,"ondisk_log_size":932,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385696,"kb_used_data":15708,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556128,"statfs":{"total":1151940427776,"available":1151545475072,"internally_reserved":0,"allocated":16084992,"data_stored":9496471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":336,"num_read_kb":418,"num_write":121,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992141"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958989+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:24:19.958989+0000","last_peered":"2024-04-26T18:24:19.958989+0000","last_clean":"2024-04-26T18:24:19.958989+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:24:19.958989+0000","last_undegraded":"2024-04-26T18:24:19.958989+0000","last_fullsized":"2024-04-26T18:24:19.958989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779276+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:24:19.779276+0000","last_peered":"2024-04-26T18:24:19.779276+0000","last_clean":"2024-04-26T18:24:19.779276+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:24:19.779276+0000","last_undegraded":"2024-04-26T18:24:19.779276+0000","last_fullsized":"2024-04-26T18:24:19.779276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778888+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:24:19.778888+0000","last_peered":"2024-04-26T18:24:19.778888+0000","last_clean":"2024-04-26T18:24:19.778888+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:24:19.778888+0000","last_undegraded":"2024-04-26T18:24:19.778888+0000","last_fullsized":"2024-04-26T18:24:19.778888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525769+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:24:19.525769+0000","last_peered":"2024-04-26T18:24:19.525769+0000","last_clean":"2024-04-26T18:24:19.525769+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:24:19.525769+0000","last_undegraded":"2024-04-26T18:24:19.525769+0000","last_fullsized":"2024-04-26T18:24:19.525769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637144+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:24:19.637144+0000","last_peered":"2024-04-26T18:24:19.637144+0000","last_clean":"2024-04-26T18:24:19.637144+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:24:19.637144+0000","last_undegraded":"2024-04-26T18:24:19.637144+0000","last_fullsized":"2024-04-26T18:24:19.637144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778407+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:24:19.778407+0000","last_peered":"2024-04-26T18:24:19.778407+0000","last_clean":"2024-04-26T18:24:19.778407+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:24:19.778407+0000","last_undegraded":"2024-04-26T18:24:19.778407+0000","last_fullsized":"2024-04-26T18:24:19.778407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628784+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:24:19.628784+0000","last_peered":"2024-04-26T18:24:19.628784+0000","last_clean":"2024-04-26T18:24:19.628784+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:24:19.628784+0000","last_undegraded":"2024-04-26T18:24:19.628784+0000","last_fullsized":"2024-04-26T18:24:19.628784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.783243+0000","last_change":"2024-04-26T18:21:58.050914+0000","last_active":"2024-04-26T18:24:19.783243+0000","last_peered":"2024-04-26T18:24:19.783243+0000","last_clean":"2024-04-26T18:24:19.783243+0000","last_became_active":"2024-04-26T18:21:58.050400+0000","last_became_peered":"2024-04-26T18:21:58.050400+0000","last_unstale":"2024-04-26T18:24:19.783243+0000","last_undegraded":"2024-04-26T18:24:19.783243+0000","last_fullsized":"2024-04-26T18:24:19.783243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:37.789853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637426+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:24:19.637426+0000","last_peered":"2024-04-26T18:24:19.637426+0000","last_clean":"2024-04-26T18:24:19.637426+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:24:19.637426+0000","last_undegraded":"2024-04-26T18:24:19.637426+0000","last_fullsized":"2024-04-26T18:24:19.637426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778659+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:24:19.778659+0000","last_peered":"2024-04-26T18:24:19.778659+0000","last_clean":"2024-04-26T18:24:19.778659+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:24:19.778659+0000","last_undegraded":"2024-04-26T18:24:19.778659+0000","last_fullsized":"2024-04-26T18:24:19.778659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525634+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:24:19.525634+0000","last_peered":"2024-04-26T18:24:19.525634+0000","last_clean":"2024-04-26T18:24:19.525634+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:24:19.525634+0000","last_undegraded":"2024-04-26T18:24:19.525634+0000","last_fullsized":"2024-04-26T18:24:19.525634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524459+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:24:19.524459+0000","last_peered":"2024-04-26T18:24:19.524459+0000","last_clean":"2024-04-26T18:24:19.524459+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:24:19.524459+0000","last_undegraded":"2024-04-26T18:24:19.524459+0000","last_fullsized":"2024-04-26T18:24:19.524459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778429+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:24:19.778429+0000","last_peered":"2024-04-26T18:24:19.778429+0000","last_clean":"2024-04-26T18:24:19.778429+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:24:19.778429+0000","last_undegraded":"2024-04-26T18:24:19.778429+0000","last_fullsized":"2024-04-26T18:24:19.778429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778395+0000","last_change":"2024-04-26T18:21:58.051267+0000","last_active":"2024-04-26T18:24:19.778395+0000","last_peered":"2024-04-26T18:24:19.778395+0000","last_clean":"2024-04-26T18:24:19.778395+0000","last_became_active":"2024-04-26T18:21:58.050912+0000","last_became_peered":"2024-04-26T18:21:58.050912+0000","last_unstale":"2024-04-26T18:24:19.778395+0000","last_undegraded":"2024-04-26T18:24:19.778395+0000","last_fullsized":"2024-04-26T18:24:19.778395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:42.244892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782460+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:24:19.782460+0000","last_peered":"2024-04-26T18:24:19.782460+0000","last_clean":"2024-04-26T18:24:19.782460+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:24:19.782460+0000","last_undegraded":"2024-04-26T18:24:19.782460+0000","last_fullsized":"2024-04-26T18:24:19.782460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.959122+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:24:19.959122+0000","last_peered":"2024-04-26T18:24:19.959122+0000","last_clean":"2024-04-26T18:24:19.959122+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:24:19.959122+0000","last_undegraded":"2024-04-26T18:24:19.959122+0000","last_fullsized":"2024-04-26T18:24:19.959122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626731+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:24:19.626731+0000","last_peered":"2024-04-26T18:24:19.626731+0000","last_clean":"2024-04-26T18:24:19.626731+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:24:19.626731+0000","last_undegraded":"2024-04-26T18:24:19.626731+0000","last_fullsized":"2024-04-26T18:24:19.626731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780323+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:24:19.780323+0000","last_peered":"2024-04-26T18:24:19.780323+0000","last_clean":"2024-04-26T18:24:19.780323+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:24:19.780323+0000","last_undegraded":"2024-04-26T18:24:19.780323+0000","last_fullsized":"2024-04-26T18:24:19.780323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626753+0000","last_change":"2024-04-26T18:21:58.059181+0000","last_active":"2024-04-26T18:24:19.626753+0000","last_peered":"2024-04-26T18:24:19.626753+0000","last_clean":"2024-04-26T18:24:19.626753+0000","last_became_active":"2024-04-26T18:21:58.051741+0000","last_became_peered":"2024-04-26T18:21:58.051741+0000","last_unstale":"2024-04-26T18:24:19.626753+0000","last_undegraded":"2024-04-26T18:24:19.626753+0000","last_fullsized":"2024-04-26T18:24:19.626753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:55.251611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629163+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:24:19.629163+0000","last_peered":"2024-04-26T18:24:19.629163+0000","last_clean":"2024-04-26T18:24:19.629163+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:24:19.629163+0000","last_undegraded":"2024-04-26T18:24:19.629163+0000","last_fullsized":"2024-04-26T18:24:19.629163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630063+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:24:19.630063+0000","last_peered":"2024-04-26T18:24:19.630063+0000","last_clean":"2024-04-26T18:24:19.630063+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:24:19.630063+0000","last_undegraded":"2024-04-26T18:24:19.630063+0000","last_fullsized":"2024-04-26T18:24:19.630063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779767+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:24:19.779767+0000","last_peered":"2024-04-26T18:24:19.779767+0000","last_clean":"2024-04-26T18:24:19.779767+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:24:19.779767+0000","last_undegraded":"2024-04-26T18:24:19.779767+0000","last_fullsized":"2024-04-26T18:24:19.779767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629262+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:24:19.629262+0000","last_peered":"2024-04-26T18:24:19.629262+0000","last_clean":"2024-04-26T18:24:19.629262+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:24:19.629262+0000","last_undegraded":"2024-04-26T18:24:19.629262+0000","last_fullsized":"2024-04-26T18:24:19.629262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782545+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:24:19.782545+0000","last_peered":"2024-04-26T18:24:19.782545+0000","last_clean":"2024-04-26T18:24:19.782545+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:24:19.782545+0000","last_undegraded":"2024-04-26T18:24:19.782545+0000","last_fullsized":"2024-04-26T18:24:19.782545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629100+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:24:19.629100+0000","last_peered":"2024-04-26T18:24:19.629100+0000","last_clean":"2024-04-26T18:24:19.629100+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:24:19.629100+0000","last_undegraded":"2024-04-26T18:24:19.629100+0000","last_fullsized":"2024-04-26T18:24:19.629100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525994+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:24:19.525994+0000","last_peered":"2024-04-26T18:24:19.525994+0000","last_clean":"2024-04-26T18:24:19.525994+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:24:19.525994+0000","last_undegraded":"2024-04-26T18:24:19.525994+0000","last_fullsized":"2024-04-26T18:24:19.525994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":220,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:20.219716+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:24:20.219716+0000","last_peered":"2024-04-26T18:24:20.219716+0000","last_clean":"2024-04-26T18:24:20.219716+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:24:20.219716+0000","last_undegraded":"2024-04-26T18:24:20.219716+0000","last_fullsized":"2024-04-26T18:24:20.219716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628856+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:24:19.628856+0000","last_peered":"2024-04-26T18:24:19.628856+0000","last_clean":"2024-04-26T18:24:19.628856+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:24:19.628856+0000","last_undegraded":"2024-04-26T18:24:19.628856+0000","last_fullsized":"2024-04-26T18:24:19.628856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637053+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:24:19.637053+0000","last_peered":"2024-04-26T18:24:19.637053+0000","last_clean":"2024-04-26T18:24:19.637053+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:24:19.637053+0000","last_undegraded":"2024-04-26T18:24:19.637053+0000","last_fullsized":"2024-04-26T18:24:19.637053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526234+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:24:19.526234+0000","last_peered":"2024-04-26T18:24:19.526234+0000","last_clean":"2024-04-26T18:24:19.526234+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:24:19.526234+0000","last_undegraded":"2024-04-26T18:24:19.526234+0000","last_fullsized":"2024-04-26T18:24:19.526234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524315+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:24:19.524315+0000","last_peered":"2024-04-26T18:24:19.524315+0000","last_clean":"2024-04-26T18:24:19.524315+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:24:19.524315+0000","last_undegraded":"2024-04-26T18:24:19.524315+0000","last_fullsized":"2024-04-26T18:24:19.524315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628948+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:24:19.628948+0000","last_peered":"2024-04-26T18:24:19.628948+0000","last_clean":"2024-04-26T18:24:19.628948+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:24:19.628948+0000","last_undegraded":"2024-04-26T18:24:19.628948+0000","last_fullsized":"2024-04-26T18:24:19.628948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525775+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:24:19.525775+0000","last_peered":"2024-04-26T18:24:19.525775+0000","last_clean":"2024-04-26T18:24:19.525775+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:24:19.525775+0000","last_undegraded":"2024-04-26T18:24:19.525775+0000","last_fullsized":"2024-04-26T18:24:19.525775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626253+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:24:19.626253+0000","last_peered":"2024-04-26T18:24:19.626253+0000","last_clean":"2024-04-26T18:24:19.626253+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:24:19.626253+0000","last_undegraded":"2024-04-26T18:24:19.626253+0000","last_fullsized":"2024-04-26T18:24:19.626253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780229+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:24:19.780229+0000","last_peered":"2024-04-26T18:24:19.780229+0000","last_clean":"2024-04-26T18:24:19.780229+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:24:19.780229+0000","last_undegraded":"2024-04-26T18:24:19.780229+0000","last_fullsized":"2024-04-26T18:24:19.780229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782966+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:24:19.782966+0000","last_peered":"2024-04-26T18:24:19.782966+0000","last_clean":"2024-04-26T18:24:19.782966+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:24:19.782966+0000","last_undegraded":"2024-04-26T18:24:19.782966+0000","last_fullsized":"2024-04-26T18:24:19.782966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778729+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:24:19.778729+0000","last_peered":"2024-04-26T18:24:19.778729+0000","last_clean":"2024-04-26T18:24:19.778729+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:24:19.778729+0000","last_undegraded":"2024-04-26T18:24:19.778729+0000","last_fullsized":"2024-04-26T18:24:19.778729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779611+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:24:19.779611+0000","last_peered":"2024-04-26T18:24:19.779611+0000","last_clean":"2024-04-26T18:24:19.779611+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:24:19.779611+0000","last_undegraded":"2024-04-26T18:24:19.779611+0000","last_fullsized":"2024-04-26T18:24:19.779611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526045+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:24:19.526045+0000","last_peered":"2024-04-26T18:24:19.526045+0000","last_clean":"2024-04-26T18:24:19.526045+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:24:19.526045+0000","last_undegraded":"2024-04-26T18:24:19.526045+0000","last_fullsized":"2024-04-26T18:24:19.526045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782568+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:24:19.782568+0000","last_peered":"2024-04-26T18:24:19.782568+0000","last_clean":"2024-04-26T18:24:19.782568+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:24:19.782568+0000","last_undegraded":"2024-04-26T18:24:19.782568+0000","last_fullsized":"2024-04-26T18:24:19.782568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":428,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:20.219700+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:24:20.219700+0000","last_peered":"2024-04-26T18:24:20.219700+0000","last_clean":"2024-04-26T18:24:20.219700+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:24:20.219700+0000","last_undegraded":"2024-04-26T18:24:20.219700+0000","last_fullsized":"2024-04-26T18:24:20.219700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":360,"num_read_kb":246,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629095+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:24:19.629095+0000","last_peered":"2024-04-26T18:24:19.629095+0000","last_clean":"2024-04-26T18:24:19.629095+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:24:19.629095+0000","last_undegraded":"2024-04-26T18:24:19.629095+0000","last_fullsized":"2024-04-26T18:24:19.629095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780221+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:24:19.780221+0000","last_peered":"2024-04-26T18:24:19.780221+0000","last_clean":"2024-04-26T18:24:19.780221+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:24:19.780221+0000","last_undegraded":"2024-04-26T18:24:19.780221+0000","last_fullsized":"2024-04-26T18:24:19.780221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524345+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:24:19.524345+0000","last_peered":"2024-04-26T18:24:19.524345+0000","last_clean":"2024-04-26T18:24:19.524345+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:24:19.524345+0000","last_undegraded":"2024-04-26T18:24:19.524345+0000","last_fullsized":"2024-04-26T18:24:19.524345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525928+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:24:19.525928+0000","last_peered":"2024-04-26T18:24:19.525928+0000","last_clean":"2024-04-26T18:24:19.525928+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:24:19.525928+0000","last_undegraded":"2024-04-26T18:24:19.525928+0000","last_fullsized":"2024-04-26T18:24:19.525928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778618+0000","last_change":"2024-04-26T18:21:58.049111+0000","last_active":"2024-04-26T18:24:19.778618+0000","last_peered":"2024-04-26T18:24:19.778618+0000","last_clean":"2024-04-26T18:24:19.778618+0000","last_became_active":"2024-04-26T18:21:58.048723+0000","last_became_peered":"2024-04-26T18:21:58.048723+0000","last_unstale":"2024-04-26T18:24:19.778618+0000","last_undegraded":"2024-04-26T18:24:19.778618+0000","last_fullsized":"2024-04-26T18:24:19.778618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:26.107528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'4","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:24.474685+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:24:24.474685+0000","last_peered":"2024-04-26T18:24:24.474685+0000","last_clean":"2024-04-26T18:24:24.474685+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:24:24.474685+0000","last_undegraded":"2024-04-26T18:24:24.474685+0000","last_fullsized":"2024-04-26T18:24:24.474685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778281+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:24:19.778281+0000","last_peered":"2024-04-26T18:24:19.778281+0000","last_clean":"2024-04-26T18:24:19.778281+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:24:19.778281+0000","last_undegraded":"2024-04-26T18:24:19.778281+0000","last_fullsized":"2024-04-26T18:24:19.778281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958688+0000","last_change":"2024-04-26T18:22:59.027035+0000","last_active":"2024-04-26T18:24:19.958688+0000","last_peered":"2024-04-26T18:24:19.958688+0000","last_clean":"2024-04-26T18:24:19.958688+0000","last_became_active":"2024-04-26T18:22:59.015160+0000","last_became_peered":"2024-04-26T18:22:59.015160+0000","last_unstale":"2024-04-26T18:24:19.958688+0000","last_undegraded":"2024-04-26T18:24:19.958688+0000","last_fullsized":"2024-04-26T18:24:19.958688+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:04.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626360+0000","last_change":"2024-04-26T18:21:58.052169+0000","last_active":"2024-04-26T18:24:19.626360+0000","last_peered":"2024-04-26T18:24:19.626360+0000","last_clean":"2024-04-26T18:24:19.626360+0000","last_became_active":"2024-04-26T18:21:58.051756+0000","last_became_peered":"2024-04-26T18:21:58.051756+0000","last_unstale":"2024-04-26T18:24:19.626360+0000","last_undegraded":"2024-04-26T18:24:19.626360+0000","last_fullsized":"2024-04-26T18:24:19.626360+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:11.730323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629270+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:24:19.629270+0000","last_peered":"2024-04-26T18:24:19.629270+0000","last_clean":"2024-04-26T18:24:19.629270+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:24:19.629270+0000","last_undegraded":"2024-04-26T18:24:19.629270+0000","last_fullsized":"2024-04-26T18:24:19.629270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958915+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:24:19.958915+0000","last_peered":"2024-04-26T18:24:19.958915+0000","last_clean":"2024-04-26T18:24:19.958915+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:24:19.958915+0000","last_undegraded":"2024-04-26T18:24:19.958915+0000","last_fullsized":"2024-04-26T18:24:19.958915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626667+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:24:19.626667+0000","last_peered":"2024-04-26T18:24:19.626667+0000","last_clean":"2024-04-26T18:24:19.626667+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:24:19.626667+0000","last_undegraded":"2024-04-26T18:24:19.626667+0000","last_fullsized":"2024-04-26T18:24:19.626667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637345+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:24:19.637345+0000","last_peered":"2024-04-26T18:24:19.637345+0000","last_clean":"2024-04-26T18:24:19.637345+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:24:19.637345+0000","last_undegraded":"2024-04-26T18:24:19.637345+0000","last_fullsized":"2024-04-26T18:24:19.637345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778652+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:24:19.778652+0000","last_peered":"2024-04-26T18:24:19.778652+0000","last_clean":"2024-04-26T18:24:19.778652+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:24:19.778652+0000","last_undegraded":"2024-04-26T18:24:19.778652+0000","last_fullsized":"2024-04-26T18:24:19.778652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637164+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:24:19.637164+0000","last_peered":"2024-04-26T18:24:19.637164+0000","last_clean":"2024-04-26T18:24:19.637164+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:24:19.637164+0000","last_undegraded":"2024-04-26T18:24:19.637164+0000","last_fullsized":"2024-04-26T18:24:19.637164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526068+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:24:19.526068+0000","last_peered":"2024-04-26T18:24:19.526068+0000","last_clean":"2024-04-26T18:24:19.526068+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:24:19.526068+0000","last_undegraded":"2024-04-26T18:24:19.526068+0000","last_fullsized":"2024-04-26T18:24:19.526068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779019+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:24:19.779019+0000","last_peered":"2024-04-26T18:24:19.779019+0000","last_clean":"2024-04-26T18:24:19.779019+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:24:19.779019+0000","last_undegraded":"2024-04-26T18:24:19.779019+0000","last_fullsized":"2024-04-26T18:24:19.779019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629371+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:24:19.629371+0000","last_peered":"2024-04-26T18:24:19.629371+0000","last_clean":"2024-04-26T18:24:19.629371+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:24:19.629371+0000","last_undegraded":"2024-04-26T18:24:19.629371+0000","last_fullsized":"2024-04-26T18:24:19.629371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525607+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:24:19.525607+0000","last_peered":"2024-04-26T18:24:19.525607+0000","last_clean":"2024-04-26T18:24:19.525607+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:24:19.525607+0000","last_undegraded":"2024-04-26T18:24:19.525607+0000","last_fullsized":"2024-04-26T18:24:19.525607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:21.633954+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:24:21.633954+0000","last_peered":"2024-04-26T18:24:21.633954+0000","last_clean":"2024-04-26T18:24:21.633954+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:24:21.633954+0000","last_undegraded":"2024-04-26T18:24:21.633954+0000","last_fullsized":"2024-04-26T18:24:21.633954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524170+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:24:19.524170+0000","last_peered":"2024-04-26T18:24:19.524170+0000","last_clean":"2024-04-26T18:24:19.524170+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:24:19.524170+0000","last_undegraded":"2024-04-26T18:24:19.524170+0000","last_fullsized":"2024-04-26T18:24:19.524170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778505+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:24:19.778505+0000","last_peered":"2024-04-26T18:24:19.778505+0000","last_clean":"2024-04-26T18:24:19.778505+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:24:19.778505+0000","last_undegraded":"2024-04-26T18:24:19.778505+0000","last_fullsized":"2024-04-26T18:24:19.778505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629160+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:24:19.629160+0000","last_peered":"2024-04-26T18:24:19.629160+0000","last_clean":"2024-04-26T18:24:19.629160+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:24:19.629160+0000","last_undegraded":"2024-04-26T18:24:19.629160+0000","last_fullsized":"2024-04-26T18:24:19.629160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524566+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:24:19.524566+0000","last_peered":"2024-04-26T18:24:19.524566+0000","last_clean":"2024-04-26T18:24:19.524566+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:24:19.524566+0000","last_undegraded":"2024-04-26T18:24:19.524566+0000","last_fullsized":"2024-04-26T18:24:19.524566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'102","reported_seq":227,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.180566+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:24:22.180566+0000","last_peered":"2024-04-26T18:24:22.180566+0000","last_clean":"2024-04-26T18:24:22.180566+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:24:22.180566+0000","last_undegraded":"2024-04-26T18:24:22.180566+0000","last_fullsized":"2024-04-26T18:24:22.180566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630066+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:24:19.630066+0000","last_peered":"2024-04-26T18:24:19.630066+0000","last_clean":"2024-04-26T18:24:19.630066+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:24:19.630066+0000","last_undegraded":"2024-04-26T18:24:19.630066+0000","last_fullsized":"2024-04-26T18:24:19.630066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525682+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:24:19.525682+0000","last_peered":"2024-04-26T18:24:19.525682+0000","last_clean":"2024-04-26T18:24:19.525682+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:24:19.525682+0000","last_undegraded":"2024-04-26T18:24:19.525682+0000","last_fullsized":"2024-04-26T18:24:19.525682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779408+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:24:19.779408+0000","last_peered":"2024-04-26T18:24:19.779408+0000","last_clean":"2024-04-26T18:24:19.779408+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:24:19.779408+0000","last_undegraded":"2024-04-26T18:24:19.779408+0000","last_fullsized":"2024-04-26T18:24:19.779408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782477+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:24:19.782477+0000","last_peered":"2024-04-26T18:24:19.782477+0000","last_clean":"2024-04-26T18:24:19.782477+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:24:19.782477+0000","last_undegraded":"2024-04-26T18:24:19.782477+0000","last_fullsized":"2024-04-26T18:24:19.782477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628827+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:24:19.628827+0000","last_peered":"2024-04-26T18:24:19.628827+0000","last_clean":"2024-04-26T18:24:19.628827+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:24:19.628827+0000","last_undegraded":"2024-04-26T18:24:19.628827+0000","last_fullsized":"2024-04-26T18:24:19.628827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628902+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:24:19.628902+0000","last_peered":"2024-04-26T18:24:19.628902+0000","last_clean":"2024-04-26T18:24:19.628902+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:24:19.628902+0000","last_undegraded":"2024-04-26T18:24:19.628902+0000","last_fullsized":"2024-04-26T18:24:19.628902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958753+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:24:19.958753+0000","last_peered":"2024-04-26T18:24:19.958753+0000","last_clean":"2024-04-26T18:24:19.958753+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:24:19.958753+0000","last_undegraded":"2024-04-26T18:24:19.958753+0000","last_fullsized":"2024-04-26T18:24:19.958753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958835+0000","last_change":"2024-04-26T18:21:58.057410+0000","last_active":"2024-04-26T18:24:19.958835+0000","last_peered":"2024-04-26T18:24:19.958835+0000","last_clean":"2024-04-26T18:24:19.958835+0000","last_became_active":"2024-04-26T18:21:58.054453+0000","last_became_peered":"2024-04-26T18:21:58.054453+0000","last_unstale":"2024-04-26T18:24:19.958835+0000","last_undegraded":"2024-04-26T18:24:19.958835+0000","last_fullsized":"2024-04-26T18:24:19.958835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:22.856834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629599+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:24:19.629599+0000","last_peered":"2024-04-26T18:24:19.629599+0000","last_clean":"2024-04-26T18:24:19.629599+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:24:19.629599+0000","last_undegraded":"2024-04-26T18:24:19.629599+0000","last_fullsized":"2024-04-26T18:24:19.629599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778674+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:24:19.778674+0000","last_peered":"2024-04-26T18:24:19.778674+0000","last_clean":"2024-04-26T18:24:19.778674+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:24:19.778674+0000","last_undegraded":"2024-04-26T18:24:19.778674+0000","last_fullsized":"2024-04-26T18:24:19.778674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525697+0000","last_change":"2024-04-26T18:21:58.069561+0000","last_active":"2024-04-26T18:24:19.525697+0000","last_peered":"2024-04-26T18:24:19.525697+0000","last_clean":"2024-04-26T18:24:19.525697+0000","last_became_active":"2024-04-26T18:21:58.053199+0000","last_became_peered":"2024-04-26T18:21:58.053199+0000","last_unstale":"2024-04-26T18:24:19.525697+0000","last_undegraded":"2024-04-26T18:24:19.525697+0000","last_fullsized":"2024-04-26T18:24:19.525697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:27.166045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778751+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:24:19.778751+0000","last_peered":"2024-04-26T18:24:19.778751+0000","last_clean":"2024-04-26T18:24:19.778751+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:24:19.778751+0000","last_undegraded":"2024-04-26T18:24:19.778751+0000","last_fullsized":"2024-04-26T18:24:19.778751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":80,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.738292+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:24:19.738292+0000","last_peered":"2024-04-26T18:24:19.738292+0000","last_clean":"2024-04-26T18:24:19.738292+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:24:19.738292+0000","last_undegraded":"2024-04-26T18:24:19.738292+0000","last_fullsized":"2024-04-26T18:24:19.738292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778454+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:24:19.778454+0000","last_peered":"2024-04-26T18:24:19.778454+0000","last_clean":"2024-04-26T18:24:19.778454+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:24:19.778454+0000","last_undegraded":"2024-04-26T18:24:19.778454+0000","last_fullsized":"2024-04-26T18:24:19.778454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524399+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:24:19.524399+0000","last_peered":"2024-04-26T18:24:19.524399+0000","last_clean":"2024-04-26T18:24:19.524399+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:24:19.524399+0000","last_undegraded":"2024-04-26T18:24:19.524399+0000","last_fullsized":"2024-04-26T18:24:19.524399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778847+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:24:19.778847+0000","last_peered":"2024-04-26T18:24:19.778847+0000","last_clean":"2024-04-26T18:24:19.778847+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:24:19.778847+0000","last_undegraded":"2024-04-26T18:24:19.778847+0000","last_fullsized":"2024-04-26T18:24:19.778847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958882+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:24:19.958882+0000","last_peered":"2024-04-26T18:24:19.958882+0000","last_clean":"2024-04-26T18:24:19.958882+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:24:19.958882+0000","last_undegraded":"2024-04-26T18:24:19.958882+0000","last_fullsized":"2024-04-26T18:24:19.958882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780084+0000","last_change":"2024-04-26T18:21:58.052815+0000","last_active":"2024-04-26T18:24:19.780084+0000","last_peered":"2024-04-26T18:24:19.780084+0000","last_clean":"2024-04-26T18:24:19.780084+0000","last_became_active":"2024-04-26T18:21:58.052405+0000","last_became_peered":"2024-04-26T18:21:58.052405+0000","last_unstale":"2024-04-26T18:24:19.780084+0000","last_undegraded":"2024-04-26T18:24:19.780084+0000","last_fullsized":"2024-04-26T18:24:19.780084+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:29.289089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780025+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:24:19.780025+0000","last_peered":"2024-04-26T18:24:19.780025+0000","last_clean":"2024-04-26T18:24:19.780025+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:24:19.780025+0000","last_undegraded":"2024-04-26T18:24:19.780025+0000","last_fullsized":"2024-04-26T18:24:19.780025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778513+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:24:19.778513+0000","last_peered":"2024-04-26T18:24:19.778513+0000","last_clean":"2024-04-26T18:24:19.778513+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:24:19.778513+0000","last_undegraded":"2024-04-26T18:24:19.778513+0000","last_fullsized":"2024-04-26T18:24:19.778513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524720+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:24:19.524720+0000","last_peered":"2024-04-26T18:24:19.524720+0000","last_clean":"2024-04-26T18:24:19.524720+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:24:19.524720+0000","last_undegraded":"2024-04-26T18:24:19.524720+0000","last_fullsized":"2024-04-26T18:24:19.524720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958902+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:24:19.958902+0000","last_peered":"2024-04-26T18:24:19.958902+0000","last_clean":"2024-04-26T18:24:19.958902+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:24:19.958902+0000","last_undegraded":"2024-04-26T18:24:19.958902+0000","last_fullsized":"2024-04-26T18:24:19.958902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780397+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:24:19.780397+0000","last_peered":"2024-04-26T18:24:19.780397+0000","last_clean":"2024-04-26T18:24:19.780397+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:24:19.780397+0000","last_undegraded":"2024-04-26T18:24:19.780397+0000","last_fullsized":"2024-04-26T18:24:19.780397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630028+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:24:19.630028+0000","last_peered":"2024-04-26T18:24:19.630028+0000","last_clean":"2024-04-26T18:24:19.630028+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:24:19.630028+0000","last_undegraded":"2024-04-26T18:24:19.630028+0000","last_fullsized":"2024-04-26T18:24:19.630028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525817+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:24:19.525817+0000","last_peered":"2024-04-26T18:24:19.525817+0000","last_clean":"2024-04-26T18:24:19.525817+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:24:19.525817+0000","last_undegraded":"2024-04-26T18:24:19.525817+0000","last_fullsized":"2024-04-26T18:24:19.525817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626199+0000","last_change":"2024-04-26T18:21:58.051597+0000","last_active":"2024-04-26T18:24:19.626199+0000","last_peered":"2024-04-26T18:24:19.626199+0000","last_clean":"2024-04-26T18:24:19.626199+0000","last_became_active":"2024-04-26T18:21:58.051139+0000","last_became_peered":"2024-04-26T18:21:58.051139+0000","last_unstale":"2024-04-26T18:24:19.626199+0000","last_undegraded":"2024-04-26T18:24:19.626199+0000","last_fullsized":"2024-04-26T18:24:19.626199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.289734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782686+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:24:19.782686+0000","last_peered":"2024-04-26T18:24:19.782686+0000","last_clean":"2024-04-26T18:24:19.782686+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:24:19.782686+0000","last_undegraded":"2024-04-26T18:24:19.782686+0000","last_fullsized":"2024-04-26T18:24:19.782686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524626+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:24:19.524626+0000","last_peered":"2024-04-26T18:24:19.524626+0000","last_clean":"2024-04-26T18:24:19.524626+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:24:19.524626+0000","last_undegraded":"2024-04-26T18:24:19.524626+0000","last_fullsized":"2024-04-26T18:24:19.524626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958776+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:24:19.958776+0000","last_peered":"2024-04-26T18:24:19.958776+0000","last_clean":"2024-04-26T18:24:19.958776+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:24:19.958776+0000","last_undegraded":"2024-04-26T18:24:19.958776+0000","last_fullsized":"2024-04-26T18:24:19.958776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596799,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1403,"num_read_kb":900,"num_write":394,"num_write_kb":1886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1276,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1790788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40520,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704632,"statfs":{"total":95995035648,"available":95953543168,"internally_reserved":0,"allocated":1044480,"data_stored":506183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645371,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28848,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716304,"statfs":{"total":95995035648,"available":95965495296,"internally_reserved":0,"allocated":1875968,"data_stored":1296602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1445888,"data_stored":899523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1433600,"data_stored":898749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1099471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495696,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":505696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":573130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":506967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40596,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704556,"statfs":{"total":95995035648,"available":95953465344,"internally_reserved":0,"allocated":1122304,"data_stored":574127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1030823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":507218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1097982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:24:28.077 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:24:28.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:24:28.332 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:29.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:29 smithi037 ceph-mon[29171]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:29.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:29 smithi037 ceph-mon[29171]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-26T18:24:29.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:29 smithi120 ceph-mon[32244]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:29.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:29 smithi120 ceph-mon[32244]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-26T18:24:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:29 smithi170 ceph-mon[32697]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:29 smithi170 ceph-mon[32697]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-04-26T18:24:30.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 -- 172.21.15.37:0/1414131324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9601094a0 msgr2=0x7fe960109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:30.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 --2- 172.21.15.37:0/1414131324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9601094a0 0x7fe960109880 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe950009990 tx=0x7fe950009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 -- 172.21.15.37:0/1414131324 shutdown_connections 2024-04-26T18:24:30.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 --2- 172.21.15.37:0/1414131324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006bad0 0x7fe9601102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 --2- 172.21.15.37:0/1414131324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9601094a0 0x7fe960109880 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 --2- 172.21.15.37:0/1414131324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe96006b130 0x7fe96006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 -- 172.21.15.37:0/1414131324 >> 172.21.15.37:0/1414131324 conn(0x7fe9600792d0 msgr2=0x7fe9600796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 -- 172.21.15.37:0/1414131324 shutdown_connections 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.998+0000 7fe965de6700 1 -- 172.21.15.37:0/1414131324 wait complete. 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 Processor -- start 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 -- start start 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006b130 0x7fe96019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9601094a0 0x7fe960199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe960113590 con 0x7fe96006bad0 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe960113410 con 0x7fe96006b130 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe965de6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe960113710 con 0x7fe9601094a0 2024-04-26T18:24:30.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe95effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe95effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53628/0 (socket says 172.21.15.37:53628) 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe95effd700 1 -- 172.21.15.37:0/779599046 learned_addr learned my addr 172.21.15.37:0/779599046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:29.999+0000 7fe95f7fe700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006b130 0x7fe96019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95ffff700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9601094a0 0x7fe960199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 -- 172.21.15.37:0/779599046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9601094a0 msgr2=0x7fe960199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9601094a0 0x7fe960199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 -- 172.21.15.37:0/779599046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006b130 msgr2=0x7fe96019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006b130 0x7fe96019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 -- 172.21.15.37:0/779599046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe960199ff0 con 0x7fe96006bad0 2024-04-26T18:24:30.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.000+0000 7fe95effd700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c9b0 tx=0x7fe95400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:30.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.001+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954007b00 con 0x7fe96006bad0 2024-04-26T18:24:30.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.001+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95400ce50 con 0x7fe96006bad0 2024-04-26T18:24:30.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.001+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96019a2e0 con 0x7fe96006bad0 2024-04-26T18:24:30.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.001+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954018660 con 0x7fe96006bad0 2024-04-26T18:24:30.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.001+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9601a65a0 con 0x7fe96006bad0 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.002+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96006a110 con 0x7fe96006bad0 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.003+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe95400f450 con 0x7fe96006bad0 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.004+0000 7fe95cff9700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 0x7fe9480a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.004+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe9540d09c0 con 0x7fe96006bad0 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.004+0000 7fe95f7fe700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 0x7fe9480a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:30.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.004+0000 7fe95f7fe700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 0x7fe9480a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe95000b5c0 tx=0x7fe950005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:30.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.008+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe954014030 con 0x7fe96006bad0 2024-04-26T18:24:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.218+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe960063860 con 0x7fe9480a3f50 2024-04-26T18:24:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.228+0000 7fe95cff9700 1 -- 172.21.15.37:0/779599046 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257776 (secure 0 0 0) 0x7fe960063860 con 0x7fe9480a3f50 2024-04-26T18:24:30.231 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:30.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 msgr2=0x7fe9480a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 0x7fe9480a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe95000b5c0 tx=0x7fe950005960 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 msgr2=0x7fe96019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c9b0 tx=0x7fe95400ccc0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 shutdown_connections 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9601094a0 0x7fe960199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe9480a3f50 0x7fe9480a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe96006b130 0x7fe96019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 --2- 172.21.15.37:0/779599046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe96006bad0 0x7fe96019fc10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 >> 172.21.15.37:0/779599046 conn(0x7fe9600792d0 msgr2=0x7fe960100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 shutdown_connections 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:30.231+0000 7fe965de6700 1 -- 172.21.15.37:0/779599046 wait complete. 2024-04-26T18:24:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T18:24:30.927 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T18:24:29.540872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384815,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1690,"num_read_kb":1270,"num_write":626,"num_write_kb":5294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1182012,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":960,"ondisk_log_size":960,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386760,"kb_used_data":16416,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555064,"statfs":{"total":1151940427776,"available":1151544385536,"internally_reserved":0,"allocated":16809984,"data_stored":10220313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197414,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":386,"num_read_kb":458,"num_write":150,"num_write_kb":917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993299"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958989+0000","last_change":"2024-04-26T18:21:08.902949+0000","last_active":"2024-04-26T18:24:19.958989+0000","last_peered":"2024-04-26T18:24:19.958989+0000","last_clean":"2024-04-26T18:24:19.958989+0000","last_became_active":"2024-04-26T18:21:08.902568+0000","last_became_peered":"2024-04-26T18:21:08.902568+0000","last_unstale":"2024-04-26T18:24:19.958989+0000","last_undegraded":"2024-04-26T18:24:19.958989+0000","last_fullsized":"2024-04-26T18:24:19.958989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:45.229337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779276+0000","last_change":"2024-04-26T18:21:08.904618+0000","last_active":"2024-04-26T18:24:19.779276+0000","last_peered":"2024-04-26T18:24:19.779276+0000","last_clean":"2024-04-26T18:24:19.779276+0000","last_became_active":"2024-04-26T18:21:08.904318+0000","last_became_peered":"2024-04-26T18:21:08.904318+0000","last_unstale":"2024-04-26T18:24:19.779276+0000","last_undegraded":"2024-04-26T18:24:19.779276+0000","last_fullsized":"2024-04-26T18:24:19.779276+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:01.530175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778888+0000","last_change":"2024-04-26T18:21:08.912331+0000","last_active":"2024-04-26T18:24:19.778888+0000","last_peered":"2024-04-26T18:24:19.778888+0000","last_clean":"2024-04-26T18:24:19.778888+0000","last_became_active":"2024-04-26T18:21:08.910251+0000","last_became_peered":"2024-04-26T18:21:08.910251+0000","last_unstale":"2024-04-26T18:24:19.778888+0000","last_undegraded":"2024-04-26T18:24:19.778888+0000","last_fullsized":"2024-04-26T18:24:19.778888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:41.766211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525769+0000","last_change":"2024-04-26T18:21:08.908362+0000","last_active":"2024-04-26T18:24:19.525769+0000","last_peered":"2024-04-26T18:24:19.525769+0000","last_clean":"2024-04-26T18:24:19.525769+0000","last_became_active":"2024-04-26T18:21:08.907607+0000","last_became_peered":"2024-04-26T18:21:08.907607+0000","last_unstale":"2024-04-26T18:24:19.525769+0000","last_undegraded":"2024-04-26T18:24:19.525769+0000","last_fullsized":"2024-04-26T18:24:19.525769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:16.737760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:08.897525+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:08.897130+0000","last_became_peered":"2024-04-26T18:21:08.897130+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:42.473178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637144+0000","last_change":"2024-04-26T18:21:08.910184+0000","last_active":"2024-04-26T18:24:19.637144+0000","last_peered":"2024-04-26T18:24:19.637144+0000","last_clean":"2024-04-26T18:24:19.637144+0000","last_became_active":"2024-04-26T18:21:08.909775+0000","last_became_peered":"2024-04-26T18:21:08.909775+0000","last_unstale":"2024-04-26T18:24:19.637144+0000","last_undegraded":"2024-04-26T18:24:19.637144+0000","last_fullsized":"2024-04-26T18:24:19.637144+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:41:47.854082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778407+0000","last_change":"2024-04-26T18:21:08.904886+0000","last_active":"2024-04-26T18:24:19.778407+0000","last_peered":"2024-04-26T18:24:19.778407+0000","last_clean":"2024-04-26T18:24:19.778407+0000","last_became_active":"2024-04-26T18:21:08.904494+0000","last_became_peered":"2024-04-26T18:21:08.904494+0000","last_unstale":"2024-04-26T18:24:19.778407+0000","last_undegraded":"2024-04-26T18:24:19.778407+0000","last_fullsized":"2024-04-26T18:24:19.778407+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:39.656984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628784+0000","last_change":"2024-04-26T18:21:08.896949+0000","last_active":"2024-04-26T18:24:19.628784+0000","last_peered":"2024-04-26T18:24:19.628784+0000","last_clean":"2024-04-26T18:24:19.628784+0000","last_became_active":"2024-04-26T18:21:08.896542+0000","last_became_peered":"2024-04-26T18:21:08.896542+0000","last_unstale":"2024-04-26T18:24:19.628784+0000","last_undegraded":"2024-04-26T18:24:19.628784+0000","last_fullsized":"2024-04-26T18:24:19.628784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:33.424244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.783243+0000","last_change":"2024-04-26T18:21:58.050914+0000","last_active":"2024-04-26T18:24:19.783243+0000","last_peered":"2024-04-26T18:24:19.783243+0000","last_clean":"2024-04-26T18:24:19.783243+0000","last_became_active":"2024-04-26T18:21:58.050400+0000","last_became_peered":"2024-04-26T18:21:58.050400+0000","last_unstale":"2024-04-26T18:24:19.783243+0000","last_undegraded":"2024-04-26T18:24:19.783243+0000","last_fullsized":"2024-04-26T18:24:19.783243+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:37.789853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637426+0000","last_change":"2024-04-26T18:21:08.907319+0000","last_active":"2024-04-26T18:24:19.637426+0000","last_peered":"2024-04-26T18:24:19.637426+0000","last_clean":"2024-04-26T18:24:19.637426+0000","last_became_active":"2024-04-26T18:21:08.906923+0000","last_became_peered":"2024-04-26T18:21:08.906923+0000","last_unstale":"2024-04-26T18:24:19.637426+0000","last_undegraded":"2024-04-26T18:24:19.637426+0000","last_fullsized":"2024-04-26T18:24:19.637426+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:01:34.448869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778659+0000","last_change":"2024-04-26T18:21:08.909424+0000","last_active":"2024-04-26T18:24:19.778659+0000","last_peered":"2024-04-26T18:24:19.778659+0000","last_clean":"2024-04-26T18:24:19.778659+0000","last_became_active":"2024-04-26T18:21:08.909091+0000","last_became_peered":"2024-04-26T18:21:08.909091+0000","last_unstale":"2024-04-26T18:24:19.778659+0000","last_undegraded":"2024-04-26T18:24:19.778659+0000","last_fullsized":"2024-04-26T18:24:19.778659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:41.301258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525634+0000","last_change":"2024-04-26T18:21:08.900434+0000","last_active":"2024-04-26T18:24:19.525634+0000","last_peered":"2024-04-26T18:24:19.525634+0000","last_clean":"2024-04-26T18:24:19.525634+0000","last_became_active":"2024-04-26T18:21:08.900106+0000","last_became_peered":"2024-04-26T18:21:08.900106+0000","last_unstale":"2024-04-26T18:24:19.525634+0000","last_undegraded":"2024-04-26T18:24:19.525634+0000","last_fullsized":"2024-04-26T18:24:19.525634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:47.112779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524459+0000","last_change":"2024-04-26T18:21:08.900903+0000","last_active":"2024-04-26T18:24:19.524459+0000","last_peered":"2024-04-26T18:24:19.524459+0000","last_clean":"2024-04-26T18:24:19.524459+0000","last_became_active":"2024-04-26T18:21:08.900580+0000","last_became_peered":"2024-04-26T18:21:08.900580+0000","last_unstale":"2024-04-26T18:24:19.524459+0000","last_undegraded":"2024-04-26T18:24:19.524459+0000","last_fullsized":"2024-04-26T18:24:19.524459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:45:06.053951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778429+0000","last_change":"2024-04-26T18:21:08.903664+0000","last_active":"2024-04-26T18:24:19.778429+0000","last_peered":"2024-04-26T18:24:19.778429+0000","last_clean":"2024-04-26T18:24:19.778429+0000","last_became_active":"2024-04-26T18:21:08.902478+0000","last_became_peered":"2024-04-26T18:21:08.902478+0000","last_unstale":"2024-04-26T18:24:19.778429+0000","last_undegraded":"2024-04-26T18:24:19.778429+0000","last_fullsized":"2024-04-26T18:24:19.778429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:00.074489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778395+0000","last_change":"2024-04-26T18:21:58.051267+0000","last_active":"2024-04-26T18:24:19.778395+0000","last_peered":"2024-04-26T18:24:19.778395+0000","last_clean":"2024-04-26T18:24:19.778395+0000","last_became_active":"2024-04-26T18:21:58.050912+0000","last_became_peered":"2024-04-26T18:21:58.050912+0000","last_unstale":"2024-04-26T18:24:19.778395+0000","last_undegraded":"2024-04-26T18:24:19.778395+0000","last_fullsized":"2024-04-26T18:24:19.778395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:42.244892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782460+0000","last_change":"2024-04-26T18:21:08.912631+0000","last_active":"2024-04-26T18:24:19.782460+0000","last_peered":"2024-04-26T18:24:19.782460+0000","last_clean":"2024-04-26T18:24:19.782460+0000","last_became_active":"2024-04-26T18:21:08.912290+0000","last_became_peered":"2024-04-26T18:21:08.912290+0000","last_unstale":"2024-04-26T18:24:19.782460+0000","last_undegraded":"2024-04-26T18:24:19.782460+0000","last_fullsized":"2024-04-26T18:24:19.782460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:52.067405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.959122+0000","last_change":"2024-04-26T18:21:08.904023+0000","last_active":"2024-04-26T18:24:19.959122+0000","last_peered":"2024-04-26T18:24:19.959122+0000","last_clean":"2024-04-26T18:24:19.959122+0000","last_became_active":"2024-04-26T18:21:08.903693+0000","last_became_peered":"2024-04-26T18:21:08.903693+0000","last_unstale":"2024-04-26T18:24:19.959122+0000","last_undegraded":"2024-04-26T18:24:19.959122+0000","last_fullsized":"2024-04-26T18:24:19.959122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:38.214570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626731+0000","last_change":"2024-04-26T18:21:08.900998+0000","last_active":"2024-04-26T18:24:19.626731+0000","last_peered":"2024-04-26T18:24:19.626731+0000","last_clean":"2024-04-26T18:24:19.626731+0000","last_became_active":"2024-04-26T18:21:08.900653+0000","last_became_peered":"2024-04-26T18:21:08.900653+0000","last_unstale":"2024-04-26T18:24:19.626731+0000","last_undegraded":"2024-04-26T18:24:19.626731+0000","last_fullsized":"2024-04-26T18:24:19.626731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:14.710528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780323+0000","last_change":"2024-04-26T18:21:08.913072+0000","last_active":"2024-04-26T18:24:19.780323+0000","last_peered":"2024-04-26T18:24:19.780323+0000","last_clean":"2024-04-26T18:24:19.780323+0000","last_became_active":"2024-04-26T18:21:08.912696+0000","last_became_peered":"2024-04-26T18:21:08.912696+0000","last_unstale":"2024-04-26T18:24:19.780323+0000","last_undegraded":"2024-04-26T18:24:19.780323+0000","last_fullsized":"2024-04-26T18:24:19.780323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:35.207848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626753+0000","last_change":"2024-04-26T18:21:58.059181+0000","last_active":"2024-04-26T18:24:19.626753+0000","last_peered":"2024-04-26T18:24:19.626753+0000","last_clean":"2024-04-26T18:24:19.626753+0000","last_became_active":"2024-04-26T18:21:58.051741+0000","last_became_peered":"2024-04-26T18:21:58.051741+0000","last_unstale":"2024-04-26T18:24:19.626753+0000","last_undegraded":"2024-04-26T18:24:19.626753+0000","last_fullsized":"2024-04-26T18:24:19.626753+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:55.251611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629163+0000","last_change":"2024-04-26T18:21:08.909254+0000","last_active":"2024-04-26T18:24:19.629163+0000","last_peered":"2024-04-26T18:24:19.629163+0000","last_clean":"2024-04-26T18:24:19.629163+0000","last_became_active":"2024-04-26T18:21:08.908633+0000","last_became_peered":"2024-04-26T18:21:08.908633+0000","last_unstale":"2024-04-26T18:24:19.629163+0000","last_undegraded":"2024-04-26T18:24:19.629163+0000","last_fullsized":"2024-04-26T18:24:19.629163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630063+0000","last_change":"2024-04-26T18:21:06.144874+0000","last_active":"2024-04-26T18:24:19.630063+0000","last_peered":"2024-04-26T18:24:19.630063+0000","last_clean":"2024-04-26T18:24:19.630063+0000","last_became_active":"2024-04-26T18:21:06.144520+0000","last_became_peered":"2024-04-26T18:21:06.144520+0000","last_unstale":"2024-04-26T18:24:19.630063+0000","last_undegraded":"2024-04-26T18:24:19.630063+0000","last_fullsized":"2024-04-26T18:24:19.630063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:40:49.427631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779767+0000","last_change":"2024-04-26T18:21:08.905516+0000","last_active":"2024-04-26T18:24:19.779767+0000","last_peered":"2024-04-26T18:24:19.779767+0000","last_clean":"2024-04-26T18:24:19.779767+0000","last_became_active":"2024-04-26T18:21:08.905175+0000","last_became_peered":"2024-04-26T18:21:08.905175+0000","last_unstale":"2024-04-26T18:24:19.779767+0000","last_undegraded":"2024-04-26T18:24:19.779767+0000","last_fullsized":"2024-04-26T18:24:19.779767+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:16.760151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629262+0000","last_change":"2024-04-26T18:21:05.879110+0000","last_active":"2024-04-26T18:24:19.629262+0000","last_peered":"2024-04-26T18:24:19.629262+0000","last_clean":"2024-04-26T18:24:19.629262+0000","last_became_active":"2024-04-26T18:21:05.878751+0000","last_became_peered":"2024-04-26T18:21:05.878751+0000","last_unstale":"2024-04-26T18:24:19.629262+0000","last_undegraded":"2024-04-26T18:24:19.629262+0000","last_fullsized":"2024-04-26T18:24:19.629262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:51:34.494938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782545+0000","last_change":"2024-04-26T18:21:08.901697+0000","last_active":"2024-04-26T18:24:19.782545+0000","last_peered":"2024-04-26T18:24:19.782545+0000","last_clean":"2024-04-26T18:24:19.782545+0000","last_became_active":"2024-04-26T18:21:08.901230+0000","last_became_peered":"2024-04-26T18:21:08.901230+0000","last_unstale":"2024-04-26T18:24:19.782545+0000","last_undegraded":"2024-04-26T18:24:19.782545+0000","last_fullsized":"2024-04-26T18:24:19.782545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:36.454413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629100+0000","last_change":"2024-04-26T18:21:05.877579+0000","last_active":"2024-04-26T18:24:19.629100+0000","last_peered":"2024-04-26T18:24:19.629100+0000","last_clean":"2024-04-26T18:24:19.629100+0000","last_became_active":"2024-04-26T18:21:05.877158+0000","last_became_peered":"2024-04-26T18:21:05.877158+0000","last_unstale":"2024-04-26T18:24:19.629100+0000","last_undegraded":"2024-04-26T18:24:19.629100+0000","last_fullsized":"2024-04-26T18:24:19.629100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:08.287130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525994+0000","last_change":"2024-04-26T18:21:08.899549+0000","last_active":"2024-04-26T18:24:19.525994+0000","last_peered":"2024-04-26T18:24:19.525994+0000","last_clean":"2024-04-26T18:24:19.525994+0000","last_became_active":"2024-04-26T18:21:08.897683+0000","last_became_peered":"2024-04-26T18:21:08.897683+0000","last_unstale":"2024-04-26T18:24:19.525994+0000","last_undegraded":"2024-04-26T18:24:19.525994+0000","last_fullsized":"2024-04-26T18:24:19.525994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:32.695249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":227,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:24.226229+0000","last_change":"2024-04-26T18:21:05.876307+0000","last_active":"2024-04-26T18:24:24.226229+0000","last_peered":"2024-04-26T18:24:24.226229+0000","last_clean":"2024-04-26T18:24:24.226229+0000","last_became_active":"2024-04-26T18:21:05.875917+0000","last_became_peered":"2024-04-26T18:21:05.875917+0000","last_unstale":"2024-04-26T18:24:24.226229+0000","last_undegraded":"2024-04-26T18:24:24.226229+0000","last_fullsized":"2024-04-26T18:24:24.226229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:55:37.291492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628856+0000","last_change":"2024-04-26T18:21:08.904753+0000","last_active":"2024-04-26T18:24:19.628856+0000","last_peered":"2024-04-26T18:24:19.628856+0000","last_clean":"2024-04-26T18:24:19.628856+0000","last_became_active":"2024-04-26T18:21:08.904270+0000","last_became_peered":"2024-04-26T18:21:08.904270+0000","last_unstale":"2024-04-26T18:24:19.628856+0000","last_undegraded":"2024-04-26T18:24:19.628856+0000","last_fullsized":"2024-04-26T18:24:19.628856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:10.886948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637053+0000","last_change":"2024-04-26T18:21:05.876626+0000","last_active":"2024-04-26T18:24:19.637053+0000","last_peered":"2024-04-26T18:24:19.637053+0000","last_clean":"2024-04-26T18:24:19.637053+0000","last_became_active":"2024-04-26T18:21:05.876205+0000","last_became_peered":"2024-04-26T18:21:05.876205+0000","last_unstale":"2024-04-26T18:24:19.637053+0000","last_undegraded":"2024-04-26T18:24:19.637053+0000","last_fullsized":"2024-04-26T18:24:19.637053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:38:40.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526234+0000","last_change":"2024-04-26T18:21:08.890809+0000","last_active":"2024-04-26T18:24:19.526234+0000","last_peered":"2024-04-26T18:24:19.526234+0000","last_clean":"2024-04-26T18:24:19.526234+0000","last_became_active":"2024-04-26T18:21:08.890387+0000","last_became_peered":"2024-04-26T18:21:08.890387+0000","last_unstale":"2024-04-26T18:24:19.526234+0000","last_undegraded":"2024-04-26T18:24:19.526234+0000","last_fullsized":"2024-04-26T18:24:19.526234+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:26:57.234501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524315+0000","last_change":"2024-04-26T18:21:05.873872+0000","last_active":"2024-04-26T18:24:19.524315+0000","last_peered":"2024-04-26T18:24:19.524315+0000","last_clean":"2024-04-26T18:24:19.524315+0000","last_became_active":"2024-04-26T18:21:05.873496+0000","last_became_peered":"2024-04-26T18:21:05.873496+0000","last_unstale":"2024-04-26T18:24:19.524315+0000","last_undegraded":"2024-04-26T18:24:19.524315+0000","last_fullsized":"2024-04-26T18:24:19.524315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:42.688179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628948+0000","last_change":"2024-04-26T18:21:08.899839+0000","last_active":"2024-04-26T18:24:19.628948+0000","last_peered":"2024-04-26T18:24:19.628948+0000","last_clean":"2024-04-26T18:24:19.628948+0000","last_became_active":"2024-04-26T18:21:08.899478+0000","last_became_peered":"2024-04-26T18:21:08.899478+0000","last_unstale":"2024-04-26T18:24:19.628948+0000","last_undegraded":"2024-04-26T18:24:19.628948+0000","last_fullsized":"2024-04-26T18:24:19.628948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:36:57.656250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525775+0000","last_change":"2024-04-26T18:21:05.878389+0000","last_active":"2024-04-26T18:24:19.525775+0000","last_peered":"2024-04-26T18:24:19.525775+0000","last_clean":"2024-04-26T18:24:19.525775+0000","last_became_active":"2024-04-26T18:21:05.878046+0000","last_became_peered":"2024-04-26T18:21:05.878046+0000","last_unstale":"2024-04-26T18:24:19.525775+0000","last_undegraded":"2024-04-26T18:24:19.525775+0000","last_fullsized":"2024-04-26T18:24:19.525775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:47.972509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626253+0000","last_change":"2024-04-26T18:21:08.904393+0000","last_active":"2024-04-26T18:24:19.626253+0000","last_peered":"2024-04-26T18:24:19.626253+0000","last_clean":"2024-04-26T18:24:19.626253+0000","last_became_active":"2024-04-26T18:21:08.904013+0000","last_became_peered":"2024-04-26T18:21:08.904013+0000","last_unstale":"2024-04-26T18:24:19.626253+0000","last_undegraded":"2024-04-26T18:24:19.626253+0000","last_fullsized":"2024-04-26T18:24:19.626253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:32.401831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":217,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:24.225572+0000","last_change":"2024-04-26T18:21:05.876754+0000","last_active":"2024-04-26T18:24:24.225572+0000","last_peered":"2024-04-26T18:24:24.225572+0000","last_clean":"2024-04-26T18:24:24.225572+0000","last_became_active":"2024-04-26T18:21:05.876349+0000","last_became_peered":"2024-04-26T18:21:05.876349+0000","last_unstale":"2024-04-26T18:24:24.225572+0000","last_undegraded":"2024-04-26T18:24:24.225572+0000","last_fullsized":"2024-04-26T18:24:24.225572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:28.328884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782966+0000","last_change":"2024-04-26T18:21:08.901078+0000","last_active":"2024-04-26T18:24:19.782966+0000","last_peered":"2024-04-26T18:24:19.782966+0000","last_clean":"2024-04-26T18:24:19.782966+0000","last_became_active":"2024-04-26T18:21:08.900760+0000","last_became_peered":"2024-04-26T18:21:08.900760+0000","last_unstale":"2024-04-26T18:24:19.782966+0000","last_undegraded":"2024-04-26T18:24:19.782966+0000","last_fullsized":"2024-04-26T18:24:19.782966+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:14.369365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778729+0000","last_change":"2024-04-26T18:21:05.876387+0000","last_active":"2024-04-26T18:24:19.778729+0000","last_peered":"2024-04-26T18:24:19.778729+0000","last_clean":"2024-04-26T18:24:19.778729+0000","last_became_active":"2024-04-26T18:21:05.875363+0000","last_became_peered":"2024-04-26T18:21:05.875363+0000","last_unstale":"2024-04-26T18:24:19.778729+0000","last_undegraded":"2024-04-26T18:24:19.778729+0000","last_fullsized":"2024-04-26T18:24:19.778729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:27.211408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779611+0000","last_change":"2024-04-26T18:21:08.912553+0000","last_active":"2024-04-26T18:24:19.779611+0000","last_peered":"2024-04-26T18:24:19.779611+0000","last_clean":"2024-04-26T18:24:19.779611+0000","last_became_active":"2024-04-26T18:21:08.910709+0000","last_became_peered":"2024-04-26T18:21:08.910709+0000","last_unstale":"2024-04-26T18:24:19.779611+0000","last_undegraded":"2024-04-26T18:24:19.779611+0000","last_fullsized":"2024-04-26T18:24:19.779611+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:01.565507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526045+0000","last_change":"2024-04-26T18:21:06.146637+0000","last_active":"2024-04-26T18:24:19.526045+0000","last_peered":"2024-04-26T18:24:19.526045+0000","last_clean":"2024-04-26T18:24:19.526045+0000","last_became_active":"2024-04-26T18:21:06.146185+0000","last_became_peered":"2024-04-26T18:21:06.146185+0000","last_unstale":"2024-04-26T18:24:19.526045+0000","last_undegraded":"2024-04-26T18:24:19.526045+0000","last_fullsized":"2024-04-26T18:24:19.526045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:10.487369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782568+0000","last_change":"2024-04-26T18:21:08.901356+0000","last_active":"2024-04-26T18:24:19.782568+0000","last_peered":"2024-04-26T18:24:19.782568+0000","last_clean":"2024-04-26T18:24:19.782568+0000","last_became_active":"2024-04-26T18:21:08.900997+0000","last_became_peered":"2024-04-26T18:21:08.900997+0000","last_unstale":"2024-04-26T18:24:19.782568+0000","last_undegraded":"2024-04-26T18:24:19.782568+0000","last_fullsized":"2024-04-26T18:24:19.782568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:47.941922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":440,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:25.227200+0000","last_change":"2024-04-26T18:21:05.878697+0000","last_active":"2024-04-26T18:24:25.227200+0000","last_peered":"2024-04-26T18:24:25.227200+0000","last_clean":"2024-04-26T18:24:25.227200+0000","last_became_active":"2024-04-26T18:21:05.877290+0000","last_became_peered":"2024-04-26T18:21:05.877290+0000","last_unstale":"2024-04-26T18:24:25.227200+0000","last_undegraded":"2024-04-26T18:24:25.227200+0000","last_fullsized":"2024-04-26T18:24:25.227200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:05:54.615633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67450,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":371,"num_read_kb":252,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629095+0000","last_change":"2024-04-26T18:21:08.908859+0000","last_active":"2024-04-26T18:24:19.629095+0000","last_peered":"2024-04-26T18:24:19.629095+0000","last_clean":"2024-04-26T18:24:19.629095+0000","last_became_active":"2024-04-26T18:21:08.908463+0000","last_became_peered":"2024-04-26T18:21:08.908463+0000","last_unstale":"2024-04-26T18:24:19.629095+0000","last_undegraded":"2024-04-26T18:24:19.629095+0000","last_fullsized":"2024-04-26T18:24:19.629095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:07.746533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780221+0000","last_change":"2024-04-26T18:21:05.876331+0000","last_active":"2024-04-26T18:24:19.780221+0000","last_peered":"2024-04-26T18:24:19.780221+0000","last_clean":"2024-04-26T18:24:19.780221+0000","last_became_active":"2024-04-26T18:21:05.875717+0000","last_became_peered":"2024-04-26T18:21:05.875717+0000","last_unstale":"2024-04-26T18:24:19.780221+0000","last_undegraded":"2024-04-26T18:24:19.780221+0000","last_fullsized":"2024-04-26T18:24:19.780221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:10.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524345+0000","last_change":"2024-04-26T18:21:08.907336+0000","last_active":"2024-04-26T18:24:19.524345+0000","last_peered":"2024-04-26T18:24:19.524345+0000","last_clean":"2024-04-26T18:24:19.524345+0000","last_became_active":"2024-04-26T18:21:08.907022+0000","last_became_peered":"2024-04-26T18:21:08.907022+0000","last_unstale":"2024-04-26T18:24:19.524345+0000","last_undegraded":"2024-04-26T18:24:19.524345+0000","last_fullsized":"2024-04-26T18:24:19.524345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:33:44.686330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525928+0000","last_change":"2024-04-26T18:21:05.876410+0000","last_active":"2024-04-26T18:24:19.525928+0000","last_peered":"2024-04-26T18:24:19.525928+0000","last_clean":"2024-04-26T18:24:19.525928+0000","last_became_active":"2024-04-26T18:21:05.876052+0000","last_became_peered":"2024-04-26T18:21:05.876052+0000","last_unstale":"2024-04-26T18:24:19.525928+0000","last_undegraded":"2024-04-26T18:24:19.525928+0000","last_fullsized":"2024-04-26T18:24:19.525928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:48:52.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778618+0000","last_change":"2024-04-26T18:21:58.049111+0000","last_active":"2024-04-26T18:24:19.778618+0000","last_peered":"2024-04-26T18:24:19.778618+0000","last_clean":"2024-04-26T18:24:19.778618+0000","last_became_active":"2024-04-26T18:21:58.048723+0000","last_became_peered":"2024-04-26T18:21:58.048723+0000","last_unstale":"2024-04-26T18:24:19.778618+0000","last_undegraded":"2024-04-26T18:24:19.778618+0000","last_fullsized":"2024-04-26T18:24:19.778618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:26.107528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'4","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:24.474685+0000","last_change":"2024-04-26T18:21:05.879127+0000","last_active":"2024-04-26T18:24:24.474685+0000","last_peered":"2024-04-26T18:24:24.474685+0000","last_clean":"2024-04-26T18:24:24.474685+0000","last_became_active":"2024-04-26T18:21:05.878748+0000","last_became_peered":"2024-04-26T18:21:05.878748+0000","last_unstale":"2024-04-26T18:24:24.474685+0000","last_undegraded":"2024-04-26T18:24:24.474685+0000","last_fullsized":"2024-04-26T18:24:24.474685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:00.168912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778281+0000","last_change":"2024-04-26T18:21:08.905758+0000","last_active":"2024-04-26T18:24:19.778281+0000","last_peered":"2024-04-26T18:24:19.778281+0000","last_clean":"2024-04-26T18:24:19.778281+0000","last_became_active":"2024-04-26T18:21:08.904136+0000","last_became_peered":"2024-04-26T18:21:08.904136+0000","last_unstale":"2024-04-26T18:24:19.778281+0000","last_undegraded":"2024-04-26T18:24:19.778281+0000","last_fullsized":"2024-04-26T18:24:19.778281+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:59.397206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958688+0000","last_change":"2024-04-26T18:22:59.027035+0000","last_active":"2024-04-26T18:24:19.958688+0000","last_peered":"2024-04-26T18:24:19.958688+0000","last_clean":"2024-04-26T18:24:19.958688+0000","last_became_active":"2024-04-26T18:22:59.015160+0000","last_became_peered":"2024-04-26T18:22:59.015160+0000","last_unstale":"2024-04-26T18:24:19.958688+0000","last_undegraded":"2024-04-26T18:24:19.958688+0000","last_fullsized":"2024-04-26T18:24:19.958688+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:07:04.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626360+0000","last_change":"2024-04-26T18:21:58.052169+0000","last_active":"2024-04-26T18:24:19.626360+0000","last_peered":"2024-04-26T18:24:19.626360+0000","last_clean":"2024-04-26T18:24:19.626360+0000","last_became_active":"2024-04-26T18:21:58.051756+0000","last_became_peered":"2024-04-26T18:21:58.051756+0000","last_unstale":"2024-04-26T18:24:19.626360+0000","last_undegraded":"2024-04-26T18:24:19.626360+0000","last_fullsized":"2024-04-26T18:24:19.626360+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:11.730323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629270+0000","last_change":"2024-04-26T18:21:05.874262+0000","last_active":"2024-04-26T18:24:19.629270+0000","last_peered":"2024-04-26T18:24:19.629270+0000","last_clean":"2024-04-26T18:24:19.629270+0000","last_became_active":"2024-04-26T18:21:05.873908+0000","last_became_peered":"2024-04-26T18:21:05.873908+0000","last_unstale":"2024-04-26T18:24:19.629270+0000","last_undegraded":"2024-04-26T18:24:19.629270+0000","last_fullsized":"2024-04-26T18:24:19.629270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:11.986166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958915+0000","last_change":"2024-04-26T18:21:08.910436+0000","last_active":"2024-04-26T18:24:19.958915+0000","last_peered":"2024-04-26T18:24:19.958915+0000","last_clean":"2024-04-26T18:24:19.958915+0000","last_became_active":"2024-04-26T18:21:08.910089+0000","last_became_peered":"2024-04-26T18:21:08.910089+0000","last_unstale":"2024-04-26T18:24:19.958915+0000","last_undegraded":"2024-04-26T18:24:19.958915+0000","last_fullsized":"2024-04-26T18:24:19.958915+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:46:53.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626667+0000","last_change":"2024-04-26T18:21:06.145483+0000","last_active":"2024-04-26T18:24:19.626667+0000","last_peered":"2024-04-26T18:24:19.626667+0000","last_clean":"2024-04-26T18:24:19.626667+0000","last_became_active":"2024-04-26T18:21:06.145096+0000","last_became_peered":"2024-04-26T18:21:06.145096+0000","last_unstale":"2024-04-26T18:24:19.626667+0000","last_undegraded":"2024-04-26T18:24:19.626667+0000","last_fullsized":"2024-04-26T18:24:19.626667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:18.208381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637345+0000","last_change":"2024-04-26T18:21:08.905531+0000","last_active":"2024-04-26T18:24:19.637345+0000","last_peered":"2024-04-26T18:24:19.637345+0000","last_clean":"2024-04-26T18:24:19.637345+0000","last_became_active":"2024-04-26T18:21:08.904594+0000","last_became_peered":"2024-04-26T18:21:08.904594+0000","last_unstale":"2024-04-26T18:24:19.637345+0000","last_undegraded":"2024-04-26T18:24:19.637345+0000","last_fullsized":"2024-04-26T18:24:19.637345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:17:49.089352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778652+0000","last_change":"2024-04-26T18:21:05.877006+0000","last_active":"2024-04-26T18:24:19.778652+0000","last_peered":"2024-04-26T18:24:19.778652+0000","last_clean":"2024-04-26T18:24:19.778652+0000","last_became_active":"2024-04-26T18:21:05.876687+0000","last_became_peered":"2024-04-26T18:21:05.876687+0000","last_unstale":"2024-04-26T18:24:19.778652+0000","last_undegraded":"2024-04-26T18:24:19.778652+0000","last_fullsized":"2024-04-26T18:24:19.778652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:16.313737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.637164+0000","last_change":"2024-04-26T18:21:08.902865+0000","last_active":"2024-04-26T18:24:19.637164+0000","last_peered":"2024-04-26T18:24:19.637164+0000","last_clean":"2024-04-26T18:24:19.637164+0000","last_became_active":"2024-04-26T18:21:08.902524+0000","last_became_peered":"2024-04-26T18:21:08.902524+0000","last_unstale":"2024-04-26T18:24:19.637164+0000","last_undegraded":"2024-04-26T18:24:19.637164+0000","last_fullsized":"2024-04-26T18:24:19.637164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:48.092396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.526068+0000","last_change":"2024-04-26T18:21:08.907194+0000","last_active":"2024-04-26T18:24:19.526068+0000","last_peered":"2024-04-26T18:24:19.526068+0000","last_clean":"2024-04-26T18:24:19.526068+0000","last_became_active":"2024-04-26T18:21:08.906843+0000","last_became_peered":"2024-04-26T18:21:08.906843+0000","last_unstale":"2024-04-26T18:24:19.526068+0000","last_undegraded":"2024-04-26T18:24:19.526068+0000","last_fullsized":"2024-04-26T18:24:19.526068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:29.990003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":217,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:24.474153+0000","last_change":"2024-04-26T18:21:06.144984+0000","last_active":"2024-04-26T18:24:24.474153+0000","last_peered":"2024-04-26T18:24:24.474153+0000","last_clean":"2024-04-26T18:24:24.474153+0000","last_became_active":"2024-04-26T18:21:06.144546+0000","last_became_peered":"2024-04-26T18:21:06.144546+0000","last_unstale":"2024-04-26T18:24:24.474153+0000","last_undegraded":"2024-04-26T18:24:24.474153+0000","last_fullsized":"2024-04-26T18:24:24.474153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:08.521023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629371+0000","last_change":"2024-04-26T18:21:08.897601+0000","last_active":"2024-04-26T18:24:19.629371+0000","last_peered":"2024-04-26T18:24:19.629371+0000","last_clean":"2024-04-26T18:24:19.629371+0000","last_became_active":"2024-04-26T18:21:08.897305+0000","last_became_peered":"2024-04-26T18:21:08.897305+0000","last_unstale":"2024-04-26T18:24:19.629371+0000","last_undegraded":"2024-04-26T18:24:19.629371+0000","last_fullsized":"2024-04-26T18:24:19.629371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:32.500113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525607+0000","last_change":"2024-04-26T18:21:05.875994+0000","last_active":"2024-04-26T18:24:19.525607+0000","last_peered":"2024-04-26T18:24:19.525607+0000","last_clean":"2024-04-26T18:24:19.525607+0000","last_became_active":"2024-04-26T18:21:05.875579+0000","last_became_peered":"2024-04-26T18:21:05.875579+0000","last_unstale":"2024-04-26T18:24:19.525607+0000","last_undegraded":"2024-04-26T18:24:19.525607+0000","last_fullsized":"2024-04-26T18:24:19.525607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:34:08.966740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:21.633954+0000","last_change":"2024-04-26T18:19:42.943214+0000","last_active":"2024-04-26T18:24:21.633954+0000","last_peered":"2024-04-26T18:24:21.633954+0000","last_clean":"2024-04-26T18:24:21.633954+0000","last_became_active":"2024-04-26T18:19:42.942734+0000","last_became_peered":"2024-04-26T18:19:42.942734+0000","last_unstale":"2024-04-26T18:24:21.633954+0000","last_undegraded":"2024-04-26T18:24:21.633954+0000","last_fullsized":"2024-04-26T18:24:21.633954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:16:21.426759+0000","last_clean_scrub_stamp":"2024-04-26T18:16:21.426759+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:34.972387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524170+0000","last_change":"2024-04-26T18:21:08.905209+0000","last_active":"2024-04-26T18:24:19.524170+0000","last_peered":"2024-04-26T18:24:19.524170+0000","last_clean":"2024-04-26T18:24:19.524170+0000","last_became_active":"2024-04-26T18:21:08.904835+0000","last_became_peered":"2024-04-26T18:21:08.904835+0000","last_unstale":"2024-04-26T18:24:19.524170+0000","last_undegraded":"2024-04-26T18:24:19.524170+0000","last_fullsized":"2024-04-26T18:24:19.524170+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:14.626673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778505+0000","last_change":"2024-04-26T18:21:05.875497+0000","last_active":"2024-04-26T18:24:19.778505+0000","last_peered":"2024-04-26T18:24:19.778505+0000","last_clean":"2024-04-26T18:24:19.778505+0000","last_became_active":"2024-04-26T18:21:05.875054+0000","last_became_peered":"2024-04-26T18:21:05.875054+0000","last_unstale":"2024-04-26T18:24:19.778505+0000","last_undegraded":"2024-04-26T18:24:19.778505+0000","last_fullsized":"2024-04-26T18:24:19.778505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:57.923756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629160+0000","last_change":"2024-04-26T18:21:06.146088+0000","last_active":"2024-04-26T18:24:19.629160+0000","last_peered":"2024-04-26T18:24:19.629160+0000","last_clean":"2024-04-26T18:24:19.629160+0000","last_became_active":"2024-04-26T18:21:06.145127+0000","last_became_peered":"2024-04-26T18:21:06.145127+0000","last_unstale":"2024-04-26T18:24:19.629160+0000","last_undegraded":"2024-04-26T18:24:19.629160+0000","last_fullsized":"2024-04-26T18:24:19.629160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:01.127281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524566+0000","last_change":"2024-04-26T18:21:08.905901+0000","last_active":"2024-04-26T18:24:19.524566+0000","last_peered":"2024-04-26T18:24:19.524566+0000","last_clean":"2024-04-26T18:24:19.524566+0000","last_became_active":"2024-04-26T18:21:08.905560+0000","last_became_peered":"2024-04-26T18:21:08.905560+0000","last_unstale":"2024-04-26T18:24:19.524566+0000","last_undegraded":"2024-04-26T18:24:19.524566+0000","last_fullsized":"2024-04-26T18:24:19.524566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:58.003996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'102","reported_seq":227,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.180566+0000","last_change":"2024-04-26T18:21:05.874523+0000","last_active":"2024-04-26T18:24:22.180566+0000","last_peered":"2024-04-26T18:24:22.180566+0000","last_clean":"2024-04-26T18:24:22.180566+0000","last_became_active":"2024-04-26T18:21:05.874133+0000","last_became_peered":"2024-04-26T18:21:05.874133+0000","last_unstale":"2024-04-26T18:24:22.180566+0000","last_undegraded":"2024-04-26T18:24:22.180566+0000","last_fullsized":"2024-04-26T18:24:22.180566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:52:12.123158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630066+0000","last_change":"2024-04-26T18:21:08.906056+0000","last_active":"2024-04-26T18:24:19.630066+0000","last_peered":"2024-04-26T18:24:19.630066+0000","last_clean":"2024-04-26T18:24:19.630066+0000","last_became_active":"2024-04-26T18:21:08.905753+0000","last_became_peered":"2024-04-26T18:21:08.905753+0000","last_unstale":"2024-04-26T18:24:19.630066+0000","last_undegraded":"2024-04-26T18:24:19.630066+0000","last_fullsized":"2024-04-26T18:24:19.630066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.144755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525682+0000","last_change":"2024-04-26T18:21:06.145125+0000","last_active":"2024-04-26T18:24:19.525682+0000","last_peered":"2024-04-26T18:24:19.525682+0000","last_clean":"2024-04-26T18:24:19.525682+0000","last_became_active":"2024-04-26T18:21:06.144746+0000","last_became_peered":"2024-04-26T18:21:06.144746+0000","last_unstale":"2024-04-26T18:24:19.525682+0000","last_undegraded":"2024-04-26T18:24:19.525682+0000","last_fullsized":"2024-04-26T18:24:19.525682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:48:19.595148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.779408+0000","last_change":"2024-04-26T18:21:08.911824+0000","last_active":"2024-04-26T18:24:19.779408+0000","last_peered":"2024-04-26T18:24:19.779408+0000","last_clean":"2024-04-26T18:24:19.779408+0000","last_became_active":"2024-04-26T18:21:08.911403+0000","last_became_peered":"2024-04-26T18:21:08.911403+0000","last_unstale":"2024-04-26T18:24:19.779408+0000","last_undegraded":"2024-04-26T18:24:19.779408+0000","last_fullsized":"2024-04-26T18:24:19.779408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:33:26.649037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782477+0000","last_change":"2024-04-26T18:21:06.145976+0000","last_active":"2024-04-26T18:24:19.782477+0000","last_peered":"2024-04-26T18:24:19.782477+0000","last_clean":"2024-04-26T18:24:19.782477+0000","last_became_active":"2024-04-26T18:21:06.145689+0000","last_became_peered":"2024-04-26T18:21:06.145689+0000","last_unstale":"2024-04-26T18:24:19.782477+0000","last_undegraded":"2024-04-26T18:24:19.782477+0000","last_fullsized":"2024-04-26T18:24:19.782477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:48:10.821579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628827+0000","last_change":"2024-04-26T18:21:08.904555+0000","last_active":"2024-04-26T18:24:19.628827+0000","last_peered":"2024-04-26T18:24:19.628827+0000","last_clean":"2024-04-26T18:24:19.628827+0000","last_became_active":"2024-04-26T18:21:08.904156+0000","last_became_peered":"2024-04-26T18:21:08.904156+0000","last_unstale":"2024-04-26T18:24:19.628827+0000","last_undegraded":"2024-04-26T18:24:19.628827+0000","last_fullsized":"2024-04-26T18:24:19.628827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:29.197470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628902+0000","last_change":"2024-04-26T18:21:05.872233+0000","last_active":"2024-04-26T18:24:19.628902+0000","last_peered":"2024-04-26T18:24:19.628902+0000","last_clean":"2024-04-26T18:24:19.628902+0000","last_became_active":"2024-04-26T18:21:05.871819+0000","last_became_peered":"2024-04-26T18:21:05.871819+0000","last_unstale":"2024-04-26T18:24:19.628902+0000","last_undegraded":"2024-04-26T18:24:19.628902+0000","last_fullsized":"2024-04-26T18:24:19.628902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:10.752302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958753+0000","last_change":"2024-04-26T18:21:08.908711+0000","last_active":"2024-04-26T18:24:19.958753+0000","last_peered":"2024-04-26T18:24:19.958753+0000","last_clean":"2024-04-26T18:24:19.958753+0000","last_became_active":"2024-04-26T18:21:08.908359+0000","last_became_peered":"2024-04-26T18:21:08.908359+0000","last_unstale":"2024-04-26T18:24:19.958753+0000","last_undegraded":"2024-04-26T18:24:19.958753+0000","last_fullsized":"2024-04-26T18:24:19.958753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:24:09.547323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958835+0000","last_change":"2024-04-26T18:21:58.057410+0000","last_active":"2024-04-26T18:24:19.958835+0000","last_peered":"2024-04-26T18:24:19.958835+0000","last_clean":"2024-04-26T18:24:19.958835+0000","last_became_active":"2024-04-26T18:21:58.054453+0000","last_became_peered":"2024-04-26T18:21:58.054453+0000","last_unstale":"2024-04-26T18:24:19.958835+0000","last_undegraded":"2024-04-26T18:24:19.958835+0000","last_fullsized":"2024-04-26T18:24:19.958835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:22.856834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.629599+0000","last_change":"2024-04-26T18:21:08.902764+0000","last_active":"2024-04-26T18:24:19.629599+0000","last_peered":"2024-04-26T18:24:19.629599+0000","last_clean":"2024-04-26T18:24:19.629599+0000","last_became_active":"2024-04-26T18:21:08.902438+0000","last_became_peered":"2024-04-26T18:21:08.902438+0000","last_unstale":"2024-04-26T18:24:19.629599+0000","last_undegraded":"2024-04-26T18:24:19.629599+0000","last_fullsized":"2024-04-26T18:24:19.629599+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:04.954484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.628906+0000","last_change":"2024-04-26T18:21:05.879248+0000","last_active":"2024-04-26T18:24:19.628906+0000","last_peered":"2024-04-26T18:24:19.628906+0000","last_clean":"2024-04-26T18:24:19.628906+0000","last_became_active":"2024-04-26T18:21:05.878908+0000","last_became_peered":"2024-04-26T18:21:05.878908+0000","last_unstale":"2024-04-26T18:24:19.628906+0000","last_undegraded":"2024-04-26T18:24:19.628906+0000","last_fullsized":"2024-04-26T18:24:19.628906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:44.723770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778674+0000","last_change":"2024-04-26T18:21:08.914352+0000","last_active":"2024-04-26T18:24:19.778674+0000","last_peered":"2024-04-26T18:24:19.778674+0000","last_clean":"2024-04-26T18:24:19.778674+0000","last_became_active":"2024-04-26T18:21:08.913968+0000","last_became_peered":"2024-04-26T18:21:08.913968+0000","last_unstale":"2024-04-26T18:24:19.778674+0000","last_undegraded":"2024-04-26T18:24:19.778674+0000","last_fullsized":"2024-04-26T18:24:19.778674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:02.990908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525697+0000","last_change":"2024-04-26T18:21:58.069561+0000","last_active":"2024-04-26T18:24:19.525697+0000","last_peered":"2024-04-26T18:24:19.525697+0000","last_clean":"2024-04-26T18:24:19.525697+0000","last_became_active":"2024-04-26T18:21:58.053199+0000","last_became_peered":"2024-04-26T18:21:58.053199+0000","last_unstale":"2024-04-26T18:24:19.525697+0000","last_undegraded":"2024-04-26T18:24:19.525697+0000","last_fullsized":"2024-04-26T18:24:19.525697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:27.166045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778751+0000","last_change":"2024-04-26T18:21:08.908844+0000","last_active":"2024-04-26T18:24:19.778751+0000","last_peered":"2024-04-26T18:24:19.778751+0000","last_clean":"2024-04-26T18:24:19.778751+0000","last_became_active":"2024-04-26T18:21:08.908343+0000","last_became_peered":"2024-04-26T18:21:08.908343+0000","last_unstale":"2024-04-26T18:24:19.778751+0000","last_undegraded":"2024-04-26T18:24:19.778751+0000","last_fullsized":"2024-04-26T18:24:19.778751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:43.724565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:21.786695+0000","last_change":"2024-04-26T18:21:05.877512+0000","last_active":"2024-04-26T18:24:21.786695+0000","last_peered":"2024-04-26T18:24:21.786695+0000","last_clean":"2024-04-26T18:24:21.786695+0000","last_became_active":"2024-04-26T18:21:05.877052+0000","last_became_peered":"2024-04-26T18:21:05.877052+0000","last_unstale":"2024-04-26T18:24:21.786695+0000","last_undegraded":"2024-04-26T18:24:21.786695+0000","last_fullsized":"2024-04-26T18:24:21.786695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:56:57.207489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778454+0000","last_change":"2024-04-26T18:21:08.905146+0000","last_active":"2024-04-26T18:24:19.778454+0000","last_peered":"2024-04-26T18:24:19.778454+0000","last_clean":"2024-04-26T18:24:19.778454+0000","last_became_active":"2024-04-26T18:21:08.904792+0000","last_became_peered":"2024-04-26T18:21:08.904792+0000","last_unstale":"2024-04-26T18:24:19.778454+0000","last_undegraded":"2024-04-26T18:24:19.778454+0000","last_fullsized":"2024-04-26T18:24:19.778454+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:52:08.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524399+0000","last_change":"2024-04-26T18:21:05.870054+0000","last_active":"2024-04-26T18:24:19.524399+0000","last_peered":"2024-04-26T18:24:19.524399+0000","last_clean":"2024-04-26T18:24:19.524399+0000","last_became_active":"2024-04-26T18:21:05.869671+0000","last_became_peered":"2024-04-26T18:21:05.869671+0000","last_unstale":"2024-04-26T18:24:19.524399+0000","last_undegraded":"2024-04-26T18:24:19.524399+0000","last_fullsized":"2024-04-26T18:24:19.524399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:04.855969+0000","last_clean_scrub_stamp":"2024-04-26T18:21:04.855969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.471924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778847+0000","last_change":"2024-04-26T18:21:08.905282+0000","last_active":"2024-04-26T18:24:19.778847+0000","last_peered":"2024-04-26T18:24:19.778847+0000","last_clean":"2024-04-26T18:24:19.778847+0000","last_became_active":"2024-04-26T18:21:08.904680+0000","last_became_peered":"2024-04-26T18:21:08.904680+0000","last_unstale":"2024-04-26T18:24:19.778847+0000","last_undegraded":"2024-04-26T18:24:19.778847+0000","last_fullsized":"2024-04-26T18:24:19.778847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:22:06.522276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958882+0000","last_change":"2024-04-26T18:21:08.910562+0000","last_active":"2024-04-26T18:24:19.958882+0000","last_peered":"2024-04-26T18:24:19.958882+0000","last_clean":"2024-04-26T18:24:19.958882+0000","last_became_active":"2024-04-26T18:21:08.909231+0000","last_became_peered":"2024-04-26T18:21:08.909231+0000","last_unstale":"2024-04-26T18:24:19.958882+0000","last_undegraded":"2024-04-26T18:24:19.958882+0000","last_fullsized":"2024-04-26T18:24:19.958882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:28.955197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780084+0000","last_change":"2024-04-26T18:21:58.052815+0000","last_active":"2024-04-26T18:24:19.780084+0000","last_peered":"2024-04-26T18:24:19.780084+0000","last_clean":"2024-04-26T18:24:19.780084+0000","last_became_active":"2024-04-26T18:21:58.052405+0000","last_became_peered":"2024-04-26T18:21:58.052405+0000","last_unstale":"2024-04-26T18:24:19.780084+0000","last_undegraded":"2024-04-26T18:24:19.780084+0000","last_fullsized":"2024-04-26T18:24:19.780084+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:29.289089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780025+0000","last_change":"2024-04-26T18:21:08.913728+0000","last_active":"2024-04-26T18:24:19.780025+0000","last_peered":"2024-04-26T18:24:19.780025+0000","last_clean":"2024-04-26T18:24:19.780025+0000","last_became_active":"2024-04-26T18:21:08.913371+0000","last_became_peered":"2024-04-26T18:21:08.913371+0000","last_unstale":"2024-04-26T18:24:19.780025+0000","last_undegraded":"2024-04-26T18:24:19.780025+0000","last_fullsized":"2024-04-26T18:24:19.780025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:11.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.778513+0000","last_change":"2024-04-26T18:21:08.912049+0000","last_active":"2024-04-26T18:24:19.778513+0000","last_peered":"2024-04-26T18:24:19.778513+0000","last_clean":"2024-04-26T18:24:19.778513+0000","last_became_active":"2024-04-26T18:21:08.910570+0000","last_became_peered":"2024-04-26T18:21:08.910570+0000","last_unstale":"2024-04-26T18:24:19.778513+0000","last_undegraded":"2024-04-26T18:24:19.778513+0000","last_fullsized":"2024-04-26T18:24:19.778513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:32:10.461663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524720+0000","last_change":"2024-04-26T18:21:08.908588+0000","last_active":"2024-04-26T18:24:19.524720+0000","last_peered":"2024-04-26T18:24:19.524720+0000","last_clean":"2024-04-26T18:24:19.524720+0000","last_became_active":"2024-04-26T18:21:08.908252+0000","last_became_peered":"2024-04-26T18:21:08.908252+0000","last_unstale":"2024-04-26T18:24:19.524720+0000","last_undegraded":"2024-04-26T18:24:19.524720+0000","last_fullsized":"2024-04-26T18:24:19.524720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:17:48.574612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958902+0000","last_change":"2024-04-26T18:21:08.904657+0000","last_active":"2024-04-26T18:24:19.958902+0000","last_peered":"2024-04-26T18:24:19.958902+0000","last_clean":"2024-04-26T18:24:19.958902+0000","last_became_active":"2024-04-26T18:21:08.904241+0000","last_became_peered":"2024-04-26T18:21:08.904241+0000","last_unstale":"2024-04-26T18:24:19.958902+0000","last_undegraded":"2024-04-26T18:24:19.958902+0000","last_fullsized":"2024-04-26T18:24:19.958902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:58.493303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.780397+0000","last_change":"2024-04-26T18:21:08.906604+0000","last_active":"2024-04-26T18:24:19.780397+0000","last_peered":"2024-04-26T18:24:19.780397+0000","last_clean":"2024-04-26T18:24:19.780397+0000","last_became_active":"2024-04-26T18:21:08.904385+0000","last_became_peered":"2024-04-26T18:21:08.904385+0000","last_unstale":"2024-04-26T18:24:19.780397+0000","last_undegraded":"2024-04-26T18:24:19.780397+0000","last_fullsized":"2024-04-26T18:24:19.780397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:10.568187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.630028+0000","last_change":"2024-04-26T18:21:08.910373+0000","last_active":"2024-04-26T18:24:19.630028+0000","last_peered":"2024-04-26T18:24:19.630028+0000","last_clean":"2024-04-26T18:24:19.630028+0000","last_became_active":"2024-04-26T18:21:08.909571+0000","last_became_peered":"2024-04-26T18:21:08.909571+0000","last_unstale":"2024-04-26T18:24:19.630028+0000","last_undegraded":"2024-04-26T18:24:19.630028+0000","last_fullsized":"2024-04-26T18:24:19.630028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:51:56.011099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.525817+0000","last_change":"2024-04-26T18:21:08.895634+0000","last_active":"2024-04-26T18:24:19.525817+0000","last_peered":"2024-04-26T18:24:19.525817+0000","last_clean":"2024-04-26T18:24:19.525817+0000","last_became_active":"2024-04-26T18:21:08.895320+0000","last_became_peered":"2024-04-26T18:21:08.895320+0000","last_unstale":"2024-04-26T18:24:19.525817+0000","last_undegraded":"2024-04-26T18:24:19.525817+0000","last_fullsized":"2024-04-26T18:24:19.525817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:16.660782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.626199+0000","last_change":"2024-04-26T18:21:58.051597+0000","last_active":"2024-04-26T18:24:19.626199+0000","last_peered":"2024-04-26T18:24:19.626199+0000","last_clean":"2024-04-26T18:24:19.626199+0000","last_became_active":"2024-04-26T18:21:58.051139+0000","last_became_peered":"2024-04-26T18:21:58.051139+0000","last_unstale":"2024-04-26T18:24:19.626199+0000","last_undegraded":"2024-04-26T18:24:19.626199+0000","last_fullsized":"2024-04-26T18:24:19.626199+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:07.289734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.782686+0000","last_change":"2024-04-26T18:21:08.910370+0000","last_active":"2024-04-26T18:24:19.782686+0000","last_peered":"2024-04-26T18:24:19.782686+0000","last_clean":"2024-04-26T18:24:19.782686+0000","last_became_active":"2024-04-26T18:21:08.910020+0000","last_became_peered":"2024-04-26T18:21:08.910020+0000","last_unstale":"2024-04-26T18:24:19.782686+0000","last_undegraded":"2024-04-26T18:24:19.782686+0000","last_fullsized":"2024-04-26T18:24:19.782686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:41:55.321056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.524626+0000","last_change":"2024-04-26T18:21:08.909318+0000","last_active":"2024-04-26T18:24:19.524626+0000","last_peered":"2024-04-26T18:24:19.524626+0000","last_clean":"2024-04-26T18:24:19.524626+0000","last_became_active":"2024-04-26T18:21:08.909003+0000","last_became_peered":"2024-04-26T18:21:08.909003+0000","last_unstale":"2024-04-26T18:24:19.524626+0000","last_undegraded":"2024-04-26T18:24:19.524626+0000","last_fullsized":"2024-04-26T18:24:19.524626+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:12.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:24:19.958776+0000","last_change":"2024-04-26T18:21:08.910202+0000","last_active":"2024-04-26T18:24:19.958776+0000","last_peered":"2024-04-26T18:24:19.958776+0000","last_clean":"2024-04-26T18:24:19.958776+0000","last_became_active":"2024-04-26T18:21:08.909850+0000","last_became_peered":"2024-04-26T18:21:08.909850+0000","last_unstale":"2024-04-26T18:24:19.958776+0000","last_undegraded":"2024-04-26T18:24:19.958776+0000","last_fullsized":"2024-04-26T18:24:19.958776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:21:07.874882+0000","last_clean_scrub_stamp":"2024-04-26T18:21:07.874882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:56.489736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":794213,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1453,"num_read_kb":940,"num_write":423,"num_write_kb":2354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1276,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2387679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383735,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40520,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704632,"statfs":{"total":95995035648,"available":95953543168,"internally_reserved":0,"allocated":1044480,"data_stored":506183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":40434116},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645372,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29160,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715992,"statfs":{"total":95995035648,"available":95965175808,"internally_reserved":0,"allocated":2072576,"data_stored":1493625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1445888,"data_stored":899523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1433600,"data_stored":898749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1099471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495697,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":505696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":573521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":506967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40596,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704556,"statfs":{"total":95995035648,"available":95953465344,"internally_reserved":0,"allocated":1122304,"data_stored":574909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28876,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716276,"statfs":{"total":95995035648,"available":95965466624,"internally_reserved":0,"allocated":1781760,"data_stored":1227455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":507218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757333,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29080,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716072,"statfs":{"total":95995035648,"available":95965257728,"internally_reserved":0,"allocated":1990656,"data_stored":1426996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:24:30.931 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:31.186 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:31.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:31 smithi037 ceph-mon[29171]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-26T18:24:31.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:31 smithi037 ceph-mon[29171]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:31 smithi120 ceph-mon[32244]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-26T18:24:31.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:31 smithi120 ceph-mon[32244]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:31 smithi170 ceph-mon[32697]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-04-26T18:24:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:31 smithi170 ceph-mon[32697]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:24:32.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 -- 172.21.15.37:0/3997257111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7988103470 msgr2=0x7f79881038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:32.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 --2- 172.21.15.37:0/3997257111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7988103470 0x7f79881038f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7970009990 tx=0x7f7970009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 -- 172.21.15.37:0/3997257111 shutdown_connections 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 --2- 172.21.15.37:0/3997257111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 0x7f7988110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 --2- 172.21.15.37:0/3997257111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79881094c0 0x7f79881098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 --2- 172.21.15.37:0/3997257111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7988103470 0x7f79881038f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 -- 172.21.15.37:0/3997257111 >> 172.21.15.37:0/3997257111 conn(0x7f798807a580 msgr2=0x7f798807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 -- 172.21.15.37:0/3997257111 shutdown_connections 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.771+0000 7f798dd7d700 1 -- 172.21.15.37:0/3997257111 wait complete. 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 Processor -- start 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 -- start start 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988103470 0x7f798819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:32.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 0x7f798819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79881135c0 con 0x7f79881094c0 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7988113440 con 0x7f7988103e30 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f798dd7d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7988113740 con 0x7f7988103470 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f7987fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f7987fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53656/0 (socket says 172.21.15.37:53656) 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f79877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988103470 0x7f798819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f7987fff700 1 -- 172.21.15.37:0/2070830167 learned_addr learned my addr 172.21.15.37:0/2070830167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.772+0000 7f7986ffd700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 0x7f798819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 -- 172.21.15.37:0/2070830167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988103470 msgr2=0x7f798819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988103470 0x7f798819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 -- 172.21.15.37:0/2070830167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 msgr2=0x7f798819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 0x7f798819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 -- 172.21.15.37:0/2070830167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798819a020 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7987fff700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f797800c900 tx=0x7f797800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79780078d0 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798819a310 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797800f450 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978018620 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.773+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a6520 con 0x7f79881094c0 2024-04-26T18:24:32.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.774+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7988104b60 con 0x7f79881094c0 2024-04-26T18:24:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.775+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7978007a70 con 0x7f79881094c0 2024-04-26T18:24:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.776+0000 7f7984ff9700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 0x7f79740a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.776+0000 7f79877fe700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 0x7f79740a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.776+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f79780d09c0 con 0x7f79881094c0 2024-04-26T18:24:32.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.777+0000 7f79877fe700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 0x7f79740a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7980007fd0 tx=0x7f79800079e0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:32.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:32.781+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7978014030 con 0x7f79881094c0 2024-04-26T18:24:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.034+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798802e490 con 0x7f79881094c0 2024-04-26T18:24:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.035+0000 7f7984ff9700 1 -- 172.21.15.37:0/2070830167 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f7978007d60 con 0x7f79881094c0 2024-04-26T18:24:33.036 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:33.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:24:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 msgr2=0x7f79740a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 0x7f79740a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7980007fd0 tx=0x7f79800079e0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 msgr2=0x7f7988199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f797800c900 tx=0x7f797800cc10 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 shutdown_connections 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79881094c0 0x7f7988199840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7988103470 0x7f798819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f79740a3ed0 0x7f79740a6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 --2- 172.21.15.37:0/2070830167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7988103e30 0x7f798819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.037+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 >> 172.21.15.37:0/2070830167 conn(0x7f798807a580 msgr2=0x7f79881020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.038+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 shutdown_connections 2024-04-26T18:24:33.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:33.038+0000 7f798dd7d700 1 -- 172.21.15.37:0/2070830167 wait complete. 2024-04-26T18:24:33.039 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:24:33.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:33 smithi037 ceph-mon[29171]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T18:24:33.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:33 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2070830167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:33 smithi120 ceph-mon[32244]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T18:24:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:33 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2070830167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:33 smithi170 ceph-mon[32697]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-26T18:24:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:33 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2070830167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:33.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:33.937 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:35.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:35 smithi037 ceph-mon[29171]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T18:24:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- 172.21.15.37:0/549760293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c103bc0 msgr2=0x7fdf2c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 --2- 172.21.15.37:0/549760293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c103fa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009900 tx=0x7fdf1c02a510 comp rx=0 tx=0).stop 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- 172.21.15.37:0/549760293 shutdown_connections 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 --2- 172.21.15.37:0/549760293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 --2- 172.21.15.37:0/549760293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 --2- 172.21.15.37:0/549760293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c103fa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- 172.21.15.37:0/549760293 >> 172.21.15.37:0/549760293 conn(0x7fdf2c0789b0 msgr2=0x7fdf2c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- 172.21.15.37:0/549760293 shutdown_connections 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- 172.21.15.37:0/549760293 wait complete. 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 Processor -- start 2024-04-26T18:24:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.407+0000 7fdf32504640 1 -- start start 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c112bb0 con 0x7fdf2c1044e0 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c112a30 con 0x7fdf2c103bc0 2024-04-26T18:24:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf32504640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c112d30 con 0x7fdf2c10cf00 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60534/0 (socket says 172.21.15.37:60534) 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 -- 172.21.15.37:0/3326465391 learned_addr learned my addr 172.21.15.37:0/3326465391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 -- 172.21.15.37:0/3326465391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 msgr2=0x7fdf2c102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 -- 172.21.15.37:0/3326465391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c103bc0 msgr2=0x7fdf2c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf2b7fe640 1 -- 172.21.15.37:0/3326465391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1a75b0 con 0x7fdf2c1044e0 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.408+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000d950 tx=0x7fdf2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf297fa640 1 -- 172.21.15.37:0/3326465391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20017070 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf297fa640 1 -- 172.21.15.37:0/3326465391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf20009c30 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1a78a0 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf297fa640 1 -- 172.21.15.37:0/3326465391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000f620 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.409+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a7df0 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.411+0000 7fdf297fa640 1 -- 172.21.15.37:0/3326465391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdf20010460 con 0x7fdf2c1044e0 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 Processor -- start 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 -- start start 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcde922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcde9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555dcddf8460 con 0x555dcde92820 2024-04-26T18:24:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dcddb2590 con 0x555dcde9d0e0 2024-04-26T18:24:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf3521a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555dcddf4420 con 0x555dcde91f00 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf297fa640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 0x7fdf140a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60542/0 (socket says 172.21.15.37:60542) 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 -- 172.21.15.37:0/582880346 learned_addr learned my addr 172.21.15.37:0/582880346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 0x7fdf140a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcde922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcde9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf297fa640 1 -- 172.21.15.37:0/3326465391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fdf200d0730 con 0x7fdf2c1044e0 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 -- 172.21.15.37:0/582880346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 msgr2=0x555dcde922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcde922e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 -- 172.21.15.37:0/582880346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 msgr2=0x555dcde9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcde9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.412+0000 7fdf2bfff640 1 -- 172.21.15.37:0/582880346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcddbf0d0 con 0x555dcde92820 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.413+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009900 tx=0x7fdf1c032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec89de40ec9f2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.413+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 0x7fdf140a63d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf24005c40 tx=0x7fdf24009500 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.413+0000 7fdf127fc640 1 -- 172.21.15.37:0/582880346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c040070 con 0x555dcde92820 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.413+0000 7fdf127fc640 1 -- 172.21.15.37:0/582880346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c004030 con 0x555dcde92820 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.413+0000 7fdf127fc640 1 -- 172.21.15.37:0/582880346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c035040 con 0x555dcde92820 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- 172.21.15.37:0/582880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 msgr2=0x555dcde9cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009900 tx=0x7fdf1c032bd0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- 172.21.15.37:0/582880346 shutdown_connections 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcde9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde92820 0x555dcde9cba0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- 172.21.15.37:0/582880346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcde922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- 172.21.15.37:0/582880346 >> 172.21.15.37:0/582880346 conn(0x555dcde8e410 msgr2=0x555dcde8e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- 172.21.15.37:0/582880346 shutdown_connections 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- 172.21.15.37:0/582880346 wait complete. 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- start start 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 Processor -- start 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcde9a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde9d0e0 0x555dcde9ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555dcde9fcb0 con 0x555dcde91f00 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555dcdd7fb60 con 0x555dcde92820 2024-04-26T18:24:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555dcde9fe30 con 0x555dcde9d0e0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.414+0000 7fdf3521a740 10 client.0 _ll_register_callbacks cb 0x555dcdda42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60550/0 (socket says 172.21.15.37:60550) 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 -- 172.21.15.37:0/277609096 learned_addr learned my addr 172.21.15.37:0/277609096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde9d0e0 0x555dcde9ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 -- 172.21.15.37:0/277609096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde9d0e0 msgr2=0x555dcde9ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde9d0e0 0x555dcde9ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 -- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 msgr2=0x555dcde9a5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcde9a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 -- 172.21.15.37:0/277609096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcdb0a210 con 0x555dcde91f00 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0052f0 tx=0x7fdf1c02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94be570465c63d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c04d060 con 0x555dcde91f00 2024-04-26T18:24:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fdf1c0043e0 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c004030 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf1c040070 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf3521a740 10 client.15264 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.415+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dcdbd6620 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c004030 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdf1c0035f0 con 0x555dcde91f00 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 20 client.15264 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdead0a0 0x555dcdeaf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 4 client.15264 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dcde06070 con 0x555dcdead0a0 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdead0a0 0x555dcdeaf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf0b7fe640 10 client.15264 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.416+0000 7fdf2b7fe640 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdead0a0 0x555dcdeaf490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6b60d01594c8e0 server_cookie=689feea215599464 in_seq=0 out_seq=0 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/277609096 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555dcde06070 con 0x555dcdead0a0 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf0b7fe640 10 client.15264 handle_command_reply: tid=0 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 client.15264 shutdown 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 msgr2=0x555dcde9a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0052f0 tx=0x7fdf1c02a510 comp rx=0 tx=0).stop 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 shutdown_connections 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdead0a0 0x555dcdeaf490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcde9a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde9d0e0 0x555dcde9ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf0b7fe640 0 client.15264 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 --2- 172.21.15.37:0/277609096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcde9a0b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.417+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 >> 172.21.15.37:0/277609096 conn(0x555dcde8e410 msgr2=0x555dcde94140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.418+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 shutdown_connections 2024-04-26T18:24:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.418+0000 7fdf3521a740 1 -- 172.21.15.37:0/277609096 wait complete. 2024-04-26T18:24:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.418+0000 7fdf3521a740 20 client.15264 trim_cache size 0 max 16384 2024-04-26T18:24:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 Processor -- start 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 -- start start 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde92820 0x555dcdeb4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcdeb4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b3070 con 0x555dcde91f00 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b0940 con 0x555dcde9d0e0 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.443+0000 7fdf3521a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1ae790 con 0x555dcde92820 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60552/0 (socket says 172.21.15.37:60552) 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 -- 172.21.15.37:0/3645715041 learned_addr learned my addr 172.21.15.37:0/3645715041 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcdeb4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 -- 172.21.15.37:0/3645715041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde92820 msgr2=0x555dcdeb4a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde92820 0x555dcdeb4a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 -- 172.21.15.37:0/3645715041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 msgr2=0x555dcdeb4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcdeb4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 -- 172.21.15.37:0/3645715041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dcde98600 con 0x555dcde91f00 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcdeb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009800 tx=0x7fdf1c032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52876b4c270e4338 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf11ffb640 1 -- 172.21.15.37:0/3645715041 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c04d040 con 0x555dcde91f00 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf11ffb640 1 -- 172.21.15.37:0/3645715041 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c0040e0 con 0x555dcde91f00 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.444+0000 7fdf11ffb640 1 -- 172.21.15.37:0/3645715041 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c035040 con 0x555dcde91f00 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- 172.21.15.37:0/3645715041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 msgr2=0x555dcdea8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009800 tx=0x7fdf1c032c50 comp rx=0 tx=0).stop 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- 172.21.15.37:0/3645715041 shutdown_connections 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde9d0e0 0x555dcdeb4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde92820 0x555dcdeb4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3645715041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde91f00 0x555dcdea8b70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- 172.21.15.37:0/3645715041 >> 172.21.15.37:0/3645715041 conn(0x555dcde8e410 msgr2=0x555dcde953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- 172.21.15.37:0/3645715041 shutdown_connections 2024-04-26T18:24:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- 172.21.15.37:0/3645715041 wait complete. 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 -- start start 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.445+0000 7fdf3521a740 1 Processor -- start 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcdea0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcdea1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1aee00 con 0x555dcde9d0e0 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b0270 con 0x555dcde92820 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1af4d0 con 0x555dcde91f00 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 10 client.0 _ll_register_callbacks cb 0x7fdf1c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60566/0 (socket says 172.21.15.37:60566) 2024-04-26T18:24:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 -- 172.21.15.37:0/3977148177 learned_addr learned my addr 172.21.15.37:0/3977148177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 -- 172.21.15.37:0/3977148177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 msgr2=0x555dcdea0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcdea0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 -- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 msgr2=0x555dcdea1060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcdea1060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 -- 172.21.15.37:0/3977148177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dcdf122d0 con 0x555dcde9d0e0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf30aa1640 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf24014780 tx=0x7fdf24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923187751aa46852 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24027070 con 0x555dcde9d0e0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fdf24004250 con 0x555dcde9d0e0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c0040e0 con 0x555dcde9d0e0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.446+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdf24011e70 con 0x555dcde9d0e0 2024-04-26T18:24:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555dcdf134f0 con 0x555dcde9d0e0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c0040e0 con 0x555dcde9d0e0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdf24020050 con 0x555dcde9d0e0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 20 client.15270 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdf13cd0 0x555dcdf16150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 4 client.15270 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555dcddffee0 con 0x555dcdf13cd0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.447+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdf13cd0 0x555dcdf16150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.448+0000 7fdf2bfff640 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdf13cd0 0x555dcdf16150 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645f3ec340a79c91 server_cookie=7a8d6458473a1109 in_seq=0 out_seq=0 2024-04-26T18:24:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.448+0000 7fdf0b7fe640 10 client.15270 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.461+0000 7fdf0b7fe640 1 -- 172.21.15.37:0/3977148177 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555dcddffee0 con 0x555dcdf13cd0 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.461+0000 7fdf0b7fe640 10 client.15270 handle_command_reply: tid=0 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 client.15270 shutdown 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 msgr2=0x555dcdea15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf24014780 tx=0x7fdf24004a40 comp rx=0 tx=0).stop 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 shutdown_connections 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555dcdf13cd0 0x555dcdf16150 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555dcde92820 0x555dcdea1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555dcde9d0e0 0x555dcdea15a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf0b7fe640 0 client.15270 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 --2- 172.21.15.37:0/3977148177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dcde91f00 0x555dcdea0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 >> 172.21.15.37:0/3977148177 conn(0x555dcde8e410 msgr2=0x555dcde8cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 shutdown_connections 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 1 -- 172.21.15.37:0/3977148177 wait complete. 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf3521a740 20 client.15270 trim_cache size 0 max 16384 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:24:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:24:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 msgr2=0x7fdf140a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 0x7fdf140a63d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf24005c40 tx=0x7fdf24009500 comp rx=0 tx=0).stop 2024-04-26T18:24:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 msgr2=0x7fdf2c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.462+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000d950 tx=0x7fdf2000de10 comp rx=0 tx=0).stop 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 shutdown_connections 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fdf140a3f10 0x7fdf140a63d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf2c10cf00 0x7fdf2c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf2c1044e0 0x7fdf2c101d50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 --2- 172.21.15.37:0/3326465391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf2c103bc0 0x7fdf2c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 >> 172.21.15.37:0/3326465391 conn(0x7fdf2c0789b0 msgr2=0x7fdf2c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 shutdown_connections 2024-04-26T18:24:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:35.463+0000 7fdf32504640 1 -- 172.21.15.37:0/3326465391 wait complete. 2024-04-26T18:24:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:35 smithi120 ceph-mon[32244]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T18:24:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:35 smithi170 ceph-mon[32697]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1657739249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 msgr2=0x7f7ecc06c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1657739249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc06c050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009990 tx=0x7f7ebc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1657739249 shutdown_connections 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1657739249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc06c590 0x7f7ecc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1657739249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1657739249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc06c050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1657739249 >> 172.21.15.37:0/1657739249 conn(0x7f7ecc0793c0 msgr2=0x7f7ecc0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1657739249 shutdown_connections 2024-04-26T18:24:35.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.617+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1657739249 wait complete. 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 Processor -- start 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 -- start start 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc06c590 0x7f7ecc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1135e0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc113460 con 0x7f7ecc1094d0 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed3afb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc113760 con 0x7f7ecc06c590 2024-04-26T18:24:35.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60584/0 (socket says 172.21.15.37:60584) 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 -- 172.21.15.37:0/1669717733 learned_addr learned my addr 172.21.15.37:0/1669717733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1096700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc06c590 0x7f7ecc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed2098700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 -- 172.21.15.37:0/1669717733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc06c590 msgr2=0x7f7ecc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc06c590 0x7f7ecc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 -- 172.21.15.37:0/1669717733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 msgr2=0x7f7ecc1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed1897700 1 -- 172.21.15.37:0/1669717733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19a040 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.618+0000 7f7ed2098700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ed1897700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800d950 tx=0x7f7ec800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8016070 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec8009df0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19a330 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800f3c0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.619+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a65a0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.620+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc06a110 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.621+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7ec8019070 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.622+0000 7f7ec2ffd700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 0x7f7eb80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.622+0000 7f7ed1096700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 0x7f7eb80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.622+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7ec80d0610 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.622+0000 7f7ed1096700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 0x7f7eb80a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00b5c0 tx=0x7f7ebc005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.626+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ec809c9f0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.880+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ecc19adc0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.881+0000 7f7ec2ffd700 1 -- 172.21.15.37:0/1669717733 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f7ec809c2f0 con 0x7f7ecc06bbd0 2024-04-26T18:24:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:35.882 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:24:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 msgr2=0x7f7eb80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 0x7f7eb80a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00b5c0 tx=0x7f7ebc005960 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 msgr2=0x7f7ecc19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800d950 tx=0x7f7ec800dc60 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 shutdown_connections 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7eb80a3f50 0x7f7eb80a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ecc1094d0 0x7f7ecc1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ecc06bbd0 0x7f7ecc19f6d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 --2- 172.21.15.37:0/1669717733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ecc06c590 0x7f7ecc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 >> 172.21.15.37:0/1669717733 conn(0x7f7ecc0793c0 msgr2=0x7f7ecc100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 shutdown_connections 2024-04-26T18:24:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:35.883+0000 7f7ed3afb700 1 -- 172.21.15.37:0/1669717733 wait complete. 2024-04-26T18:24:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:24:36.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:36 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1669717733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:36.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:36 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1669717733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:36.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:24:36.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:36 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1669717733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:37.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:37 smithi037 ceph-mon[29171]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-26T18:24:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:37 smithi120 ceph-mon[32244]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-26T18:24:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:37 smithi170 ceph-mon[32697]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 -- 172.21.15.37:0/405248901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c41094c0 msgr2=0x7fe5c41098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/405248901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c41094c0 0x7fe5c41098a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009990 tx=0x7fe5b4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 -- 172.21.15.37:0/405248901 shutdown_connections 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/405248901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c4103e30 0x7fe5c4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/405248901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c41094c0 0x7fe5c41098a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/405248901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 -- 172.21.15.37:0/405248901 >> 172.21.15.37:0/405248901 conn(0x7fe5c407a580 msgr2=0x7fe5c407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 -- 172.21.15.37:0/405248901 shutdown_connections 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.416+0000 7fe5c9f26700 1 -- 172.21.15.37:0/405248901 wait complete. 2024-04-26T18:24:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 Processor -- start 2024-04-26T18:24:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 -- start start 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c41094c0 0x7fe5c4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4113610 con 0x7fe5c4103e30 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4113490 con 0x7fe5c4103470 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c9f26700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4113790 con 0x7fe5c41094c0 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60596/0 (socket says 172.21.15.37:60596) 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c2ffd700 1 -- 172.21.15.37:0/1832308010 learned_addr learned my addr 172.21.15.37:0/1832308010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c3fff700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c41094c0 0x7fe5c4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.417+0000 7fe5c37fe700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 -- 172.21.15.37:0/1832308010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c41094c0 msgr2=0x7fe5c4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c41094c0 0x7fe5c4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 -- 172.21.15.37:0/1832308010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 msgr2=0x7fe5c419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 -- 172.21.15.37:0/1832308010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c419a0b0 con 0x7fe5c4103e30 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c37fe700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c2ffd700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d950 tx=0x7fe5b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8016070 con 0x7fe5c4103e30 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5b8009df0 con 0x7fe5c4103e30 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c419a3a0 con 0x7fe5c4103e30 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800f3c0 con 0x7fe5c4103e30 2024-04-26T18:24:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.418+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a6660 con 0x7fe5c4103e30 2024-04-26T18:24:38.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.419+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c406a110 con 0x7fe5c4103e30 2024-04-26T18:24:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.420+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe5b8019070 con 0x7fe5c4103e30 2024-04-26T18:24:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.421+0000 7fe5c0ff9700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 0x7fe5ac0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.421+0000 7fe5c37fe700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 0x7fe5ac0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.421+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe5b80d0700 con 0x7fe5c4103e30 2024-04-26T18:24:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.422+0000 7fe5c37fe700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 0x7fe5ac0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006010 tx=0x7fe5b400b540 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.426+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5b809cae0 con 0x7fe5c4103e30 2024-04-26T18:24:38.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.634+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5c402e490 con 0x7fe5c4103e30 2024-04-26T18:24:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.636+0000 7fe5c0ff9700 1 -- 172.21.15.37:0/1832308010 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23101 (secure 0 0 0) 0x7fe5b809c3e0 con 0x7fe5c4103e30 2024-04-26T18:24:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:38.636 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":93,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:24:19.519790+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3120809922":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1405864097":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3994756262":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/2586849717":"2024-04-27T18:24:19.519750+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:6800/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:24:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 msgr2=0x7fe5ac0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 0x7fe5ac0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006010 tx=0x7fe5b400b540 comp rx=0 tx=0).stop 2024-04-26T18:24:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 msgr2=0x7fe5c419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800d950 tx=0x7fe5b800dc60 comp rx=0 tx=0).stop 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 shutdown_connections 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c41094c0 0x7fe5c4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe5ac0a3ed0 0x7fe5ac0a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5c4103470 0x7fe5c419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 --2- 172.21.15.37:0/1832308010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c4103e30 0x7fe5c419fcd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 >> 172.21.15.37:0/1832308010 conn(0x7fe5c407a580 msgr2=0x7fe5c4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 shutdown_connections 2024-04-26T18:24:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:38.638+0000 7fe5c9f26700 1 -- 172.21.15.37:0/1832308010 wait complete. 2024-04-26T18:24:39.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:39 smithi037 ceph-mon[29171]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:24:39.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:39 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1832308010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:39.314 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T18:24:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:39 smithi120 ceph-mon[32244]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:24:39.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:39 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1832308010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:39 smithi170 ceph-mon[32697]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:24:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:39 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1832308010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:41.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:41 smithi037 ceph-mon[29171]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:24:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:41 smithi120 ceph-mon[32244]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:24:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:41 smithi170 ceph-mon[32697]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:24:41.749 INFO:teuthology.orchestra.run.smithi037.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T18:24:41.749 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T18:24:41.749 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9496 2024-04-26T18:24:41.750 DEBUG:tasks.quiescer:Yielding 2024-04-26T18:24:41.750 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T18:24:41.761 INFO:tasks.workunit:Pulling workunits from ref bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:24:41.762 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T18:24:41.762 INFO:tasks.workunit:timeout=3h 2024-04-26T18:24:41.762 INFO:tasks.workunit:cleanup=True 2024-04-26T18:24:41.762 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:41.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:41.779 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:41.779 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T18:24:41.779 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 68h/104d Inode: 1099511628283 Links: 2 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 18:21:42.313322429 +0000 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 18:21:42.313322429 +0000 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 18:24:17.202792719 +0000 2024-04-26T18:24:41.780 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 18:21:42.313322429 +0000 2024-04-26T18:24:41.780 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T18:24:41.780 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T18:24:41.852 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:24:41.907 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T18:24:42.016 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:43.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:43 smithi037 ceph-mon[29171]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-26T18:24:43.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:43 smithi120 ceph-mon[32244]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-26T18:24:43.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 -- 172.21.15.37:0/3035567312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 msgr2=0x7f06701038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:43.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 --2- 172.21.15.37:0/3035567312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f06701038f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0664009990 tx=0x7f0664009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 -- 172.21.15.37:0/3035567312 shutdown_connections 2024-04-26T18:24:43.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 --2- 172.21.15.37:0/3035567312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670103e30 0x7f0670110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 --2- 172.21.15.37:0/3035567312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06701094c0 0x7f06701098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.562+0000 7f06753c9700 1 --2- 172.21.15.37:0/3035567312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f06701038f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.563+0000 7f06753c9700 1 -- 172.21.15.37:0/3035567312 >> 172.21.15.37:0/3035567312 conn(0x7f067007a580 msgr2=0x7f067007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.563+0000 7f06753c9700 1 -- 172.21.15.37:0/3035567312 shutdown_connections 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.563+0000 7f06753c9700 1 -- 172.21.15.37:0/3035567312 wait complete. 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.563+0000 7f06753c9700 1 Processor -- start 2024-04-26T18:24:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.563+0000 7f06753c9700 1 -- start start 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0670103e30 0x7f0670075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06701094c0 0x7f06700761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0670113640 con 0x7f0670103470 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06701134c0 con 0x7f0670103e30 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f06753c9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f06701137c0 con 0x7f06701094c0 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60604/0 (socket says 172.21.15.37:60604) 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 -- 172.21.15.37:0/1061696860 learned_addr learned my addr 172.21.15.37:0/1061696860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066e7fc700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0670103e30 0x7f0670075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066f7fe700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06701094c0 0x7f06700761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:43.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 -- 172.21.15.37:0/1061696860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06701094c0 msgr2=0x7f06700761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06701094c0 0x7f06700761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 -- 172.21.15.37:0/1061696860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0670103e30 msgr2=0x7f0670075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0670103e30 0x7f0670075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 -- 172.21.15.37:0/1061696860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06701a80a0 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f066effd700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f066000d980 tx=0x7f066000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.564+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660016070 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.565+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0660004830 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.565+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06701a8390 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.565+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06600054c0 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.565+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701a88a0 con 0x7f0670103470 2024-04-26T18:24:43.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.565+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0670104b60 con 0x7f0670103470 2024-04-26T18:24:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.567+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0660004220 con 0x7f0670103470 2024-04-26T18:24:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.567+0000 7f0657fff700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 0x7f06580a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.567+0000 7f066e7fc700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 0x7f06580a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.567+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f06600d0de0 con 0x7f0670103470 2024-04-26T18:24:43.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.568+0000 7f066e7fc700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 0x7f06580a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f066400b5c0 tx=0x7f0664005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.571+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f066001f020 con 0x7f0670103470 2024-04-26T18:24:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:43 smithi170 ceph-mon[32697]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-26T18:24:43.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.826+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0670076c50 con 0x7f0670103470 2024-04-26T18:24:43.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.826+0000 7f0657fff700 1 -- 172.21.15.37:0/1061696860 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f066009d220 con 0x7f0670103470 2024-04-26T18:24:43.827 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:24:43.827 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.828+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 msgr2=0x7f06580a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.828+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 0x7f06580a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f066400b5c0 tx=0x7f0664005960 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 msgr2=0x7f0670075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f066000d980 tx=0x7f066000dc90 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 shutdown_connections 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06701094c0 0x7f06700761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0670103470 0x7f0670075720 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f06580a3ed0 0x7f06580a6390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 --2- 172.21.15.37:0/1061696860 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0670103e30 0x7f0670075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:43.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 >> 172.21.15.37:0/1061696860 conn(0x7f067007a580 msgr2=0x7f06701022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:43.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 shutdown_connections 2024-04-26T18:24:43.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:43.829+0000 7f06753c9700 1 -- 172.21.15.37:0/1061696860 wait complete. 2024-04-26T18:24:43.830 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:24:44.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:44 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1061696860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:44.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:21:10.996859+0000', 'modified': '2024-04-26T18:24:17.933921+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24464, 'mds_1': 24449, 'mds_2': 24442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/707687219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 707687219}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 707687219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/3605086601', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3605086601}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 3605086601}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/2099915678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 2099915678}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2099915678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6839/1569501885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 1569501885}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 1569501885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.120:6837/1266100832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1266100832}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1266100832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.120:6839/1246417570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 1246417570}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 1246417570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24464, 'qdb_cluster': [24464, 24449, 24442]} 2024-04-26T18:24:44.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:24:44.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:24:44.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:44 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1061696860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:44.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:44 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1061696860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:44.743 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:45.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:45 smithi037 ceph-mon[29171]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:24:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:45 smithi120 ceph-mon[32244]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:24:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:45 smithi170 ceph-mon[32697]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 -- 172.21.15.37:0/2436629895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114108390 msgr2=0x7f0114108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 --2- 172.21.15.37:0/2436629895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114108390 0x7f0114108770 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f011000b0a0 tx=0x7f011000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 -- 172.21.15.37:0/2436629895 shutdown_connections 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 --2- 172.21.15.37:0/2436629895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114102ce0 0x7f011410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 --2- 172.21.15.37:0/2436629895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114108390 0x7f0114108770 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 --2- 172.21.15.37:0/2436629895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102320 0x7f01141027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 -- 172.21.15.37:0/2436629895 >> 172.21.15.37:0/2436629895 conn(0x7f01140792d0 msgr2=0x7f01140796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 -- 172.21.15.37:0/2436629895 shutdown_connections 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.425+0000 7f011b4c8700 1 -- 172.21.15.37:0/2436629895 wait complete. 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 Processor -- start 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 -- start start 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102ce0 0x7f01141a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114108390 0x7f011419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01141124b0 con 0x7f0114102320 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0114112330 con 0x7f0114108390 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f011b4c8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0114112630 con 0x7f0114102ce0 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45486/0 (socket says 172.21.15.37:45486) 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119264700 1 -- 172.21.15.37:0/371363051 learned_addr learned my addr 172.21.15.37:0/371363051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:46.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0118a63700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102ce0 0x7f01141a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119264700 1 -- 172.21.15.37:0/371363051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102ce0 msgr2=0x7f01141a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119264700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102ce0 0x7f01141a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.426+0000 7f0119a65700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114108390 0x7f011419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f0119264700 1 -- 172.21.15.37:0/371363051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114108390 msgr2=0x7f011419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f0119264700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114108390 0x7f011419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f0119264700 1 -- 172.21.15.37:0/371363051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011419c510 con 0x7f0114102320 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f0119264700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f011000b070 tx=0x7f0110004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f010a7fc700 1 -- 172.21.15.37:0/371363051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110021070 con 0x7f0114102320 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f010a7fc700 1 -- 172.21.15.37:0/371363051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0110007e30 con 0x7f0114102320 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011419c7a0 con 0x7f0114102320 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f010a7fc700 1 -- 172.21.15.37:0/371363051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110004840 con 0x7f0114102320 2024-04-26T18:24:46.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.427+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a8890 con 0x7f0114102320 2024-04-26T18:24:46.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.429+0000 7f010a7fc700 1 -- 172.21.15.37:0/371363051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0110007980 con 0x7f0114102320 2024-04-26T18:24:46.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.430+0000 7f010a7fc700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 0x7f01000a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.430+0000 7f010a7fc700 1 -- 172.21.15.37:0/371363051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f011000f020 con 0x7f0114102320 2024-04-26T18:24:46.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.430+0000 7f0118a63700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 0x7f01000a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0118a63700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 0x7f01000a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f011410f610 tx=0x7f010400b410 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 Processor -- start 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 -- start start 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c818eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c812cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c7656f0 con 0x55dc1c8132d0 2024-04-26T18:24:46.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.431+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c704590 con 0x55dc1c8193f0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c7616b0 con 0x55dc1c818ad0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45502/0 (socket says 172.21.15.37:45502) 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 -- 172.21.15.37:0/1143139741 learned_addr learned my addr 172.21.15.37:0/1143139741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119264700 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c818eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0118a63700 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c812cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 -- 172.21.15.37:0/1143139741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 msgr2=0x55dc1c818eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c818eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 -- 172.21.15.37:0/1143139741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 msgr2=0x55dc1c812cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c812cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 -- 172.21.15.37:0/1143139741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1c6b2880 con 0x55dc1c8132d0 2024-04-26T18:24:46.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f0119a65700 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f010c009af0 tx=0x7f010c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3289a40c8abc8840 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.432+0000 7f00ff7fe700 1 -- 172.21.15.37:0/1143139741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c016070 con 0x55dc1c8132d0 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f00ff7fe700 1 -- 172.21.15.37:0/1143139741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010c005020 con 0x55dc1c8132d0 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f00ff7fe700 1 -- 172.21.15.37:0/1143139741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ea40 con 0x55dc1c8132d0 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1143139741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 msgr2=0x55dc1c813710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f010c009af0 tx=0x7f010c009e00 comp rx=0 tx=0).stop 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1143139741 shutdown_connections 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c813710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c818eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1143139741 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c812cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.433+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1143139741 >> 172.21.15.37:0/1143139741 conn(0x55dc1c73fcf0 msgr2=0x55dc1c80e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1143139741 shutdown_connections 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1143139741 wait complete. 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- start start 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 Processor -- start 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c818ad0 0x55dc1c81c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c8193f0 0x55dc1c81c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c81ff10 con 0x55dc1c8132d0 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c6cb900 con 0x55dc1c818ad0 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.434+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c820090 con 0x55dc1c8193f0 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0125d5ab80 10 client.0 _ll_register_callbacks cb 0x55dc1c6f9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45514/0 (socket says 172.21.15.37:45514) 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 -- 172.21.15.37:0/3723258462 learned_addr learned my addr 172.21.15.37:0/3723258462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0118a63700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c818ad0 0x55dc1c81c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119a65700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c8193f0 0x55dc1c81c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 -- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c8193f0 msgr2=0x55dc1c81c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c8193f0 0x55dc1c81c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 -- 172.21.15.37:0/3723258462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c818ad0 msgr2=0x55dc1c81c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c818ad0 0x55dc1c81c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 -- 172.21.15.37:0/3723258462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1c81cfd0 con 0x55dc1c8132d0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0119264700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01100095c0 tx=0x7f011000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18262c3c83ac1969 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f00fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100a1070 con 0x55dc1c8132d0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0110004000 con 0x55dc1c8132d0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.435+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010c005020 con 0x55dc1c8132d0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f01100cfe60 con 0x55dc1c8132d0 2024-04-26T18:24:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 10 client.15300 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc1c82a930 con 0x55dc1c8132d0 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010c005020 con 0x55dc1c8132d0 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0110011070 con 0x55dc1c8132d0 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 20 client.15300 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c82cfd0 0x55dc1c82e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 4 client.15300 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc1c7730d0 con 0x55dc1c82cfd0 2024-04-26T18:24:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.436+0000 7f0118a63700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c82cfd0 0x55dc1c82e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.437+0000 7f0118a63700 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c82cfd0 0x55dc1c82e440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64907c4d016104a8 server_cookie=2bbb673209fb1a9c in_seq=0 out_seq=0 2024-04-26T18:24:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.437+0000 7f00fcff9700 10 client.15300 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f00fcff9700 1 -- 172.21.15.37:0/3723258462 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc1c7730d0 con 0x55dc1c82cfd0 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f00fcff9700 10 client.15300 handle_command_reply: tid=0 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 client.15300 shutdown 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 msgr2=0x55dc1c81f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01100095c0 tx=0x7f011000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 shutdown_connections 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c8193f0 0x55dc1c81c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c81f100 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c82cfd0 0x55dc1c82e440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/3723258462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c818ad0 0x55dc1c81c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 >> 172.21.15.37:0/3723258462 conn(0x55dc1c73fcf0 msgr2=0x55dc1c80d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f00fcff9700 0 client.15300 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 shutdown_connections 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 1 -- 172.21.15.37:0/3723258462 wait complete. 2024-04-26T18:24:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.438+0000 7f0125d5ab80 20 client.15300 trim_cache size 0 max 16384 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 Processor -- start 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 -- start start 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c829360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c82bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82c780 con 0x55dc1c8132d0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82ce50 con 0x55dc1c8193f0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82d520 con 0x55dc1c818ad0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45528/0 (socket says 172.21.15.37:45528) 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0119264700 1 -- 172.21.15.37:0/1037739978 learned_addr learned my addr 172.21.15.37:0/1037739978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0119a65700 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c82bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.459+0000 7f0118a63700 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c829360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 -- 172.21.15.37:0/1037739978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 msgr2=0x55dc1c829360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c829360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 -- 172.21.15.37:0/1037739978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 msgr2=0x55dc1c82bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c82bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 -- 172.21.15.37:0/1037739978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc1c81a9c0 con 0x55dc1c8132d0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f0119264700 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0110009fd0 tx=0x7f01100042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a362665494d6661 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f00feffd700 1 -- 172.21.15.37:0/1037739978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100a1070 con 0x55dc1c8132d0 2024-04-26T18:24:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f00feffd700 1 -- 172.21.15.37:0/1037739978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011009e000 con 0x55dc1c8132d0 2024-04-26T18:24:46.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.460+0000 7f00feffd700 1 -- 172.21.15.37:0/1037739978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100a2150 con 0x55dc1c8132d0 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1037739978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 msgr2=0x55dc1c812cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0110009fd0 tx=0x7f01100042e0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1037739978 shutdown_connections 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c82bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c812cb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1037739978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c829360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1037739978 >> 172.21.15.37:0/1037739978 conn(0x55dc1c73fcf0 msgr2=0x55dc1c816620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1037739978 shutdown_connections 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1037739978 wait complete. 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 -- start start 2024-04-26T18:24:46.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.461+0000 7f0125d5ab80 1 Processor -- start 2024-04-26T18:24:46.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c828350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c828890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82e2c0 con 0x55dc1c8132d0 2024-04-26T18:24:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82dbf0 con 0x55dc1c8193f0 2024-04-26T18:24:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dc1c82e990 con 0x55dc1c818ad0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 10 client.0 _ll_register_callbacks cb 0x55dc1c6f9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45534/0 (socket says 172.21.15.37:45534) 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 -- 172.21.15.37:0/1975998599 learned_addr learned my addr 172.21.15.37:0/1975998599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0118a63700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c828350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119a65700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c828890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 -- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 msgr2=0x55dc1c828350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c828350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 -- 172.21.15.37:0/1975998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 msgr2=0x55dc1c828890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c828890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 -- 172.21.15.37:0/1975998599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc1c829010 con 0x55dc1c8132d0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0119264700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f01100a6450 tx=0x7f011009e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b221b6bebe4e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f00fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100a1070 con 0x55dc1c8132d0 2024-04-26T18:24:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f01100a66f0 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.462+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011009e000 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f01100a2150 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 10 client.15306 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc1c829010 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f011009e000 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0110011070 con 0x55dc1c8132d0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 20 client.15306 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c898480 0x55dc1c897190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 4 client.15306 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 --> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] -- command(tid 0: {"prefix": "status"}) -- 0x55dc1c76d170 con 0x55dc1c898480 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.463+0000 7f0118a63700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c898480 0x55dc1c897190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:46.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0118a63700 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c898480 0x55dc1c897190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=481ca83c8260e78e server_cookie=81b10c9896a1fe28 in_seq=0 out_seq=0 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f00fcff9700 10 client.15306 ms_handle_connect on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f00fcff9700 1 -- 172.21.15.37:0/1975998599 <== mds.2 v2:172.21.15.170:6838/1569501885 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55dc1c76d170 con 0x55dc1c898480 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f00fcff9700 10 client.15306 handle_command_reply: tid=0 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 client.15306 shutdown 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 msgr2=0x55dc1c837b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f01100a6450 tx=0x7f011009e4a0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 shutdown_connections 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:6838/1569501885,v1:172.21.15.170:6839/1569501885] conn(0x55dc1c898480 0x55dc1c897190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dc1c8193f0 0x55dc1c828890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dc1c8132d0 0x55dc1c837b80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 --2- 172.21.15.37:0/1975998599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dc1c818ad0 0x55dc1c828350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 >> 172.21.15.37:0/1975998599 conn(0x55dc1c73fcf0 msgr2=0x55dc1c82bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f00fcff9700 0 client.15306 ms_handle_reset on v2:172.21.15.170:6838/1569501885 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 shutdown_connections 2024-04-26T18:24:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 1 -- 172.21.15.37:0/1975998599 wait complete. 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.464+0000 7f0125d5ab80 20 client.15306 trim_cache size 0 max 16384 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24442, 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 201.52344733499999, 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 93, 2024-04-26T18:24:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 241.58191456 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 msgr2=0x7f01000a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 0x7f01000a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f011410f610 tx=0x7f010400b410 comp rx=0 tx=0).stop 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 msgr2=0x7f01141a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f011000b070 tx=0x7f0110004d00 comp rx=0 tx=0).stop 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 shutdown_connections 2024-04-26T18:24:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f01000a3ed0 0x7f01000a6390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0114108390 0x7f011419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0114102320 0x7f01141a0910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 --2- 172.21.15.37:0/371363051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0114102ce0 0x7f01141a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 >> 172.21.15.37:0/371363051 conn(0x7f01140792d0 msgr2=0x7f01140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 shutdown_connections 2024-04-26T18:24:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:46.465+0000 7f011b4c8700 1 -- 172.21.15.37:0/371363051 wait complete. 2024-04-26T18:24:47.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:24:47.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:47 smithi037 ceph-mon[29171]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-26T18:24:47.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:47 smithi120 ceph-mon[32244]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-26T18:24:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:47 smithi170 ceph-mon[32697]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 -- 172.21.15.37:0/2968056019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec108920 msgr2=0x7fe6ec108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/2968056019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec108920 0x7fe6ec108d00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009930 tx=0x7fe6e402a420 comp rx=0 tx=0).stop 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 -- 172.21.15.37:0/2968056019 shutdown_connections 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/2968056019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec10f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/2968056019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec102910 0x7fe6ec102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/2968056019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec108920 0x7fe6ec108d00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 -- 172.21.15.37:0/2968056019 >> 172.21.15.37:0/2968056019 conn(0x7fe6ec079dc0 msgr2=0x7fe6ec07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 -- 172.21.15.37:0/2968056019 shutdown_connections 2024-04-26T18:24:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.559+0000 7fe6f44e1640 1 -- 172.21.15.37:0/2968056019 wait complete. 2024-04-26T18:24:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 Processor -- start 2024-04-26T18:24:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 -- start start 2024-04-26T18:24:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec108920 0x7fe6ec19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112c30 con 0x7fe6ec102910 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112ab0 con 0x7fe6ec1032d0 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f44e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec112db0 con 0x7fe6ec108920 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45552/0 (socket says 172.21.15.37:45552) 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.560+0000 7fe6f227d640 1 -- 172.21.15.37:0/563900644 learned_addr learned my addr 172.21.15.37:0/563900644 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec108920 0x7fe6ec19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 -- 172.21.15.37:0/563900644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec108920 msgr2=0x7fe6ec19d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec108920 0x7fe6ec19d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 -- 172.21.15.37:0/563900644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 msgr2=0x7fe6ec199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 -- 172.21.15.37:0/563900644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec19e120 con 0x7fe6ec102910 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f227d640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4031440 tx=0x7fe6e4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6eb7fe640 1 -- 172.21.15.37:0/563900644 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e403f070 con 0x7fe6ec102910 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6eb7fe640 1 -- 172.21.15.37:0/563900644 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e4031d60 con 0x7fe6ec102910 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6eb7fe640 1 -- 172.21.15.37:0/563900644 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4035030 con 0x7fe6ec102910 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec19e3b0 con 0x7fe6ec102910 2024-04-26T18:24:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.561+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec1a5ca0 con 0x7fe6ec102910 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.563+0000 7fe6eb7fe640 1 -- 172.21.15.37:0/563900644 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe6e4042070 con 0x7fe6ec102910 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 Processor -- start 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 -- start start 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a99405dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a993fb9460 con 0x55a994052f00 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a993f73590 con 0x55a99405e0e0 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a993fb5420 con 0x55a994053820 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.564+0000 7fe6f2a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45564/0 (socket says 172.21.15.37:45564) 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/3927525111 learned_addr learned my addr 172.21.15.37:0/3927525111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f227d640 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a99405dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6eb7fe640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 0x7fe6d40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/3927525111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 msgr2=0x55a99405dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a99405dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/3927525111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 msgr2=0x55a9940604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/3927525111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a993f800d0 con 0x55a994052f00 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 0x7fe6d40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6eb7fe640 1 -- 172.21.15.37:0/563900644 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe6e40f1350 con 0x7fe6ec102910 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00a2b0 tx=0x7fe6dc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cf139e70e88cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6e8bfe640 1 -- 172.21.15.37:0/3927525111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc022070 con 0x55a994052f00 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6e8bfe640 1 -- 172.21.15.37:0/3927525111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc018070 con 0x55a994052f00 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.565+0000 7fe6e8bfe640 1 -- 172.21.15.37:0/3927525111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc01cc60 con 0x55a994052f00 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.566+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 0x7fe6d40a6420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc01f000 tx=0x7fe6dc020000 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.566+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3927525111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 msgr2=0x55a9940532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.566+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00a2b0 tx=0x7fe6dc00e7e0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3927525111 shutdown_connections 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a99405dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3927525111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a9940532e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3927525111 >> 172.21.15.37:0/3927525111 conn(0x55a99404f410 msgr2=0x55a99404f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3927525111 shutdown_connections 2024-04-26T18:24:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3927525111 wait complete. 2024-04-26T18:24:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- start start 2024-04-26T18:24:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 Processor -- start 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a99405b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a99405e0e0 0x55a99405ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.567+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a994060d10 con 0x55a994052f00 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a993f40b60 con 0x55a994053820 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a994060e90 con 0x55a99405e0e0 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f71ff740 10 client.0 _ll_register_callbacks cb 0x55a993f652b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45576/0 (socket says 172.21.15.37:45576) 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 -- 172.21.15.37:0/3419135002 learned_addr learned my addr 172.21.15.37:0/3419135002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a99405b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a99405e0e0 0x55a99405ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 -- 172.21.15.37:0/3419135002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a99405e0e0 msgr2=0x55a99405ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a99405e0e0 0x55a99405ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 -- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 msgr2=0x55a99405b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a99405b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 -- 172.21.15.37:0/3419135002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a993ccb210 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f227d640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40bd090 tx=0x7fe6e403b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cf79c9994d016c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4049050 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe6e40c58e0 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc018070 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.568+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6e40bced0 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 10 client.15318 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a993d97620 con 0x55a994052f00 2024-04-26T18:24:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc018070 con 0x55a994052f00 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe6e40d1ba0 con 0x55a994052f00 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 10 client.15318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 20 client.15318 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a99406e1a0 0x55a994070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 4 client.15318 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a993fc7070 con 0x55a99406e1a0 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.569+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a99406e1a0 0x55a994070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.570+0000 7fe6d1ffb640 10 client.15318 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.570+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a99406e1a0 0x55a994070640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e03c57bdb264cc1 server_cookie=1aee1c0a212dd7d0 in_seq=0 out_seq=0 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.570+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/3419135002 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a993fc7070 con 0x55a99406e1a0 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.570+0000 7fe6d1ffb640 10 client.15318 handle_command_reply: tid=0 2024-04-26T18:24:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 client.15318 shutdown 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 msgr2=0x55a99405afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40bd090 tx=0x7fe6e403b9b0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 shutdown_connections 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a99406e1a0 0x55a994070640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a99405b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a99405e0e0 0x55a99405ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6d1ffb640 0 client.15318 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/3419135002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a99405afa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 >> 172.21.15.37:0/3419135002 conn(0x55a99404f410 msgr2=0x55a994055140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 shutdown_connections 2024-04-26T18:24:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 1 -- 172.21.15.37:0/3419135002 wait complete. 2024-04-26T18:24:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.571+0000 7fe6f71ff740 20 client.15318 trim_cache size 0 max 16384 2024-04-26T18:24:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 Processor -- start 2024-04-26T18:24:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 -- start start 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994052f00 0x55a994054fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a994075f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1b0d20 con 0x55a99405e0e0 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1ae5f0 con 0x55a994053820 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1ac440 con 0x55a994052f00 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.586+0000 7fe6f2a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45590/0 (socket says 172.21.15.37:45590) 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/2768658778 learned_addr learned my addr 172.21.15.37:0/2768658778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a994075f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f227d640 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994052f00 0x55a994054fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/2768658778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994052f00 msgr2=0x55a994054fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994052f00 0x55a994054fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/2768658778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 msgr2=0x55a994075f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a994075f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 -- 172.21.15.37:0/2768658778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99404ad60 con 0x55a99405e0e0 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005fc0 tx=0x7fe6dc00a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ada544cbc6da71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6d3fff640 1 -- 172.21.15.37:0/2768658778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc022040 con 0x55a99405e0e0 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6d3fff640 1 -- 172.21.15.37:0/2768658778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc018040 con 0x55a99405e0e0 2024-04-26T18:24:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.587+0000 7fe6d3fff640 1 -- 172.21.15.37:0/2768658778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc004280 con 0x55a99405e0e0 2024-04-26T18:24:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- 172.21.15.37:0/2768658778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 msgr2=0x55a994076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc005fc0 tx=0x7fe6dc00a7b0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- 172.21.15.37:0/2768658778 shutdown_connections 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a994053820 0x55a994075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a99405e0e0 0x55a994076450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/2768658778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994052f00 0x55a994054fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- 172.21.15.37:0/2768658778 >> 172.21.15.37:0/2768658778 conn(0x55a99404f410 msgr2=0x55a9940548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- 172.21.15.37:0/2768658778 shutdown_connections 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- 172.21.15.37:0/2768658778 wait complete. 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 -- start start 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.588+0000 7fe6f71ff740 1 Processor -- start 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a994066670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1acab0 con 0x55a994052f00 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1adf20 con 0x55a99405e0e0 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec1ad180 con 0x55a994053820 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 10 client.0 _ll_register_callbacks cb 0x7fe6e4005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45592/0 (socket says 172.21.15.37:45592) 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 -- 172.21.15.37:0/1523767895 learned_addr learned my addr 172.21.15.37:0/1523767895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f2a7e640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a994066670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 -- 172.21.15.37:0/1523767895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 msgr2=0x55a994066670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a994066670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 -- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 msgr2=0x55a9940626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 -- 172.21.15.37:0/1523767895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a994062eb0 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f227d640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40c5840 tx=0x7fe6e403b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e14d6c63beebdec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4049070 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe6e4031ac0 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.589+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc018040 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6e40bca60 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a9940d5430 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6dc018040 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe6e40eeb90 con 0x55a994052f00 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 20 client.15324 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9940d59a0 0x55a9940d7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 4 client.15324 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a993fc0ee0 con 0x55a9940d59a0 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.590+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9940d59a0 0x55a9940d7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.591+0000 7fe6f1a7c640 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9940d59a0 0x55a9940d7dc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a0e105dfa8280e server_cookie=cee3631ea976a705 in_seq=0 out_seq=0 2024-04-26T18:24:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.591+0000 7fe6d1ffb640 10 client.15324 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6d1ffb640 1 -- 172.21.15.37:0/1523767895 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a993fc0ee0 con 0x55a9940d59a0 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6d1ffb640 10 client.15324 handle_command_reply: tid=0 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 client.15324 shutdown 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 msgr2=0x55a994066130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40c5840 tx=0x7fe6e403b840 comp rx=0 tx=0).stop 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 shutdown_connections 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9940d59a0 0x55a9940d7dc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a99405e0e0 0x55a9940626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a994053820 0x55a994066670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6d1ffb640 0 client.15324 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 --2- 172.21.15.37:0/1523767895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a994052f00 0x55a994066130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.599+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 >> 172.21.15.37:0/1523767895 conn(0x55a99404f410 msgr2=0x55a99405af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 shutdown_connections 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f71ff740 1 -- 172.21.15.37:0/1523767895 wait complete. 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f71ff740 20 client.15324 trim_cache size 0 max 16384 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:24:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:24:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 msgr2=0x7fe6d40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 0x7fe6d40a6420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc01f000 tx=0x7fe6dc020000 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 msgr2=0x7fe6ec199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4031440 tx=0x7fe6e4003d50 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 shutdown_connections 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe6d40a3f60 0x7fe6d40a6420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6ec1032d0 0x7fe6ec199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe6ec108920 0x7fe6ec19d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 --2- 172.21.15.37:0/563900644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6ec102910 0x7fe6ec199130 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 >> 172.21.15.37:0/563900644 conn(0x7fe6ec079dc0 msgr2=0x7fe6ec100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.600+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 shutdown_connections 2024-04-26T18:24:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:24:48.601+0000 7fe6f44e1640 1 -- 172.21.15.37:0/563900644 wait complete. 2024-04-26T18:24:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 -- 172.21.15.37:0/2071166121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454102ce0 msgr2=0x7f945410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 --2- 172.21.15.37:0/2071166121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454102ce0 0x7f945410f1d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b0a0 tx=0x7f944c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 -- 172.21.15.37:0/2071166121 shutdown_connections 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 --2- 172.21.15.37:0/2071166121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454102ce0 0x7f945410f1d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 --2- 172.21.15.37:0/2071166121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454108390 0x7f9454108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 --2- 172.21.15.37:0/2071166121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102320 0x7f94541027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 -- 172.21.15.37:0/2071166121 >> 172.21.15.37:0/2071166121 conn(0x7f94540792d0 msgr2=0x7f94540796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 -- 172.21.15.37:0/2071166121 shutdown_connections 2024-04-26T18:24:49.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.024+0000 7f945a146700 1 -- 172.21.15.37:0/2071166121 wait complete. 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 Processor -- start 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 -- start start 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454102320 0x7f9454104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102ce0 0x7f9454104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112530 con 0x7f9454108390 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94541123b0 con 0x7f9454102ce0 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f945a146700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f94541126b0 con 0x7f9454102320 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45616/0 (socket says 172.21.15.37:45616) 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f9452ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102ce0 0x7f9454104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f9453fff700 1 -- 172.21.15.37:0/4103468049 learned_addr learned my addr 172.21.15.37:0/4103468049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:49.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.025+0000 7f94537fe700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454102320 0x7f9454104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 -- 172.21.15.37:0/4103468049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454102320 msgr2=0x7f9454104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454102320 0x7f9454104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 -- 172.21.15.37:0/4103468049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102ce0 msgr2=0x7f9454104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102ce0 0x7f9454104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 -- 172.21.15.37:0/4103468049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94541a8090 con 0x7f9454108390 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9453fff700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b830 tx=0x7f944c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9450ff9700 1 -- 172.21.15.37:0/4103468049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c021070 con 0x7f9454108390 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9450ff9700 1 -- 172.21.15.37:0/4103468049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c007e30 con 0x7f9454108390 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f9450ff9700 1 -- 172.21.15.37:0/4103468049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c004700 con 0x7f9454108390 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541a8320 con 0x7f9454108390 2024-04-26T18:24:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.026+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a87d0 con 0x7f9454108390 2024-04-26T18:24:49.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.029+0000 7f9450ff9700 1 -- 172.21.15.37:0/4103468049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f944c007980 con 0x7f9454108390 2024-04-26T18:24:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.030+0000 7f9450ff9700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 0x7f943c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.030+0000 7f9450ff9700 1 -- 172.21.15.37:0/4103468049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f944c00f020 con 0x7f9454108390 2024-04-26T18:24:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.030+0000 7f94537fe700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 0x7f943c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.030+0000 7f94537fe700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 0x7f943c0a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9444009a10 tx=0x7f9444009190 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.030+0000 7f94649d8b80 1 Processor -- start 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 -- start start 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dc2a20 0x562dc0dc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0dbd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dc0d0f7e0 con 0x562dc0dbd6f0 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562dc0c758e0 con 0x562dc0dc2a20 2024-04-26T18:24:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94649d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562dc0d0b7a0 con 0x562dc0dbcd50 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45626/0 (socket says 172.21.15.37:45626) 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 -- 172.21.15.37:0/669874953 learned_addr learned my addr 172.21.15.37:0/669874953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f94537fe700 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dc2a20 0x562dc0dc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9452ffd700 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0dbd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 -- 172.21.15.37:0/669874953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 msgr2=0x562dc0dbd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0dbd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 -- 172.21.15.37:0/669874953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dc2a20 msgr2=0x562dc0dc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dc2a20 0x562dc0dc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.031+0000 7f9453fff700 1 -- 172.21.15.37:0/669874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc0c5c880 con 0x562dc0dbd6f0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f9453fff700 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f944c09daa0 tx=0x7f944c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ada129bdb1f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f9439ffb700 1 -- 172.21.15.37:0/669874953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c0cfe60 con 0x562dc0dbd6f0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f9439ffb700 1 -- 172.21.15.37:0/669874953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c09e4a0 con 0x562dc0dbd6f0 2024-04-26T18:24:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f9439ffb700 1 -- 172.21.15.37:0/669874953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c02ddd0 con 0x562dc0dbd6f0 2024-04-26T18:24:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f94649d8b80 1 -- 172.21.15.37:0/669874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 msgr2=0x562dc0dc9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f94649d8b80 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f944c09daa0 tx=0x7f944c0099e0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f94649d8b80 1 -- 172.21.15.37:0/669874953 shutdown_connections 2024-04-26T18:24:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.032+0000 7f94649d8b80 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc9770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dc2a20 0x562dc0dc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 --2- 172.21.15.37:0/669874953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0dbd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- 172.21.15.37:0/669874953 >> 172.21.15.37:0/669874953 conn(0x562dc0ceb570 msgr2=0x562dc0db7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- 172.21.15.37:0/669874953 shutdown_connections 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- 172.21.15.37:0/669874953 wait complete. 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- start start 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 Processor -- start 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0dc93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0dc6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dc3420 con 0x562dc0dbd6f0 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562dc0cc9a30 con 0x562dc0dbcd50 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.033+0000 7f94649d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dc9cb0 con 0x562dc0dc2a20 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f94649d8b80 10 client.0 _ll_register_callbacks cb 0x562dc0ca7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:49.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9452ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9452ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45636/0 (socket says 172.21.15.37:45636) 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0dc6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9452ffd700 1 -- 172.21.15.37:0/4013815636 learned_addr learned my addr 172.21.15.37:0/4013815636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f94537fe700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0dc93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 -- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 msgr2=0x562dc0dc93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0dc93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 -- 172.21.15.37:0/4013815636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 msgr2=0x562dc0dc6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 -- 172.21.15.37:0/4013815636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc0ccd430 con 0x562dc0dc2a20 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9452ffd700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9453fff700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0dc6cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f944c09df60 tx=0x7f944c09a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e9433ad0d66453 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9432ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:24:49.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c0cfe60 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c09e4a0 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.034+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f944c0074e0 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f944c0a2150 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 10 client.24985 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562dc0dd4240 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c09e4a0 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f944c011070 con 0x562dc0dc2a20 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 10 client.24985 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 10 client.24985 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 20 client.24985 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:49.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0dd65b0 0x562dc0dd7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 4 client.24985 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:49.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc0d1d1c0 con 0x562dc0dd65b0 2024-04-26T18:24:49.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.035+0000 7f94537fe700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0dd65b0 0x562dc0dd7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.036+0000 7f9432ffd700 10 client.24985 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:49.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.036+0000 7f94537fe700 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0dd65b0 0x562dc0dd7a20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16cb0fb07fbdcab server_cookie=6eaff7c20aa79ddc in_seq=0 out_seq=0 2024-04-26T18:24:49.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f9432ffd700 1 -- 172.21.15.37:0/4013815636 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dc0d1d1c0 con 0x562dc0dd65b0 2024-04-26T18:24:49.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f9432ffd700 10 client.24985 handle_command_reply: tid=0 2024-04-26T18:24:49.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 client.24985 shutdown 2024-04-26T18:24:49.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 msgr2=0x562dc0dc6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0dc6cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f944c09df60 tx=0x7f944c09a110 comp rx=0 tx=0).stop 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 shutdown_connections 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0dd65b0 0x562dc0dd7a20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0dc6cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0dc93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 --2- 172.21.15.37:0/4013815636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0dc6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f9432ffd700 0 client.24985 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 >> 172.21.15.37:0/4013815636 conn(0x562dc0ceb570 msgr2=0x562dc0dbbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 shutdown_connections 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 1 -- 172.21.15.37:0/4013815636 wait complete. 2024-04-26T18:24:49.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.037+0000 7f94649d8b80 20 client.24985 trim_cache size 0 max 16384 2024-04-26T18:24:49.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 Processor -- start 2024-04-26T18:24:49.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 -- start start 2024-04-26T18:24:49.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0db22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 0x562dc0db77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 0x562dc0dbbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd5d60 con 0x562dc0dc2a20 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd6430 con 0x562dc0dbd6f0 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94649d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd6b00 con 0x562dc0dbcd50 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 0x562dc0dbbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f9453fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 0x562dc0dbbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45640/0 (socket says 172.21.15.37:45640) 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0db22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f9453fff700 1 -- 172.21.15.37:0/3874013993 learned_addr learned my addr 172.21.15.37:0/3874013993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f9452ffd700 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 0x562dc0db77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94537fe700 1 -- 172.21.15.37:0/3874013993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 msgr2=0x562dc0db77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.058+0000 7f94537fe700 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 0x562dc0db77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94537fe700 1 -- 172.21.15.37:0/3874013993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 msgr2=0x562dc0dbbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94537fe700 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 0x562dc0dbbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94537fe700 1 -- 172.21.15.37:0/3874013993 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc0dc33d0 con 0x562dc0dbcd50 2024-04-26T18:24:49.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f9452ffd700 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 0x562dc0db77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94537fe700 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0db22b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x562dc0dd8ba0 tx=0x7f9444015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d549b5e061d3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94397fa700 1 -- 172.21.15.37:0/3874013993 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944401b040 con 0x562dc0dbcd50 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94397fa700 1 -- 172.21.15.37:0/3874013993 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944402f070 con 0x562dc0dbcd50 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.059+0000 7f94397fa700 1 -- 172.21.15.37:0/3874013993 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444015480 con 0x562dc0dbcd50 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- 172.21.15.37:0/3874013993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 msgr2=0x562dc0db22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0db22b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x562dc0dd8ba0 tx=0x7f9444015000 comp rx=0 tx=0).stop 2024-04-26T18:24:49.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- 172.21.15.37:0/3874013993 shutdown_connections 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dc2a20 0x562dc0dbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dbcd50 0x562dc0db22b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 --2- 172.21.15.37:0/3874013993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbd6f0 0x562dc0db77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- 172.21.15.37:0/3874013993 >> 172.21.15.37:0/3874013993 conn(0x562dc0ceb570 msgr2=0x562dc0dd5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- 172.21.15.37:0/3874013993 shutdown_connections 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- 172.21.15.37:0/3874013993 wait complete. 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 -- start start 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.060+0000 7f94649d8b80 1 Processor -- start 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0ddae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0ddf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd78a0 con 0x562dc0dbd6f0 2024-04-26T18:24:49.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd71d0 con 0x562dc0dbcd50 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562dc0dd7f70 con 0x562dc0dc2a20 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94649d8b80 10 client.0 _ll_register_callbacks cb 0x562dc0ca7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45646/0 (socket says 172.21.15.37:45646) 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 -- 172.21.15.37:0/2366561837 learned_addr learned my addr 172.21.15.37:0/2366561837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94537fe700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0ddae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9453fff700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0ddf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 -- 172.21.15.37:0/2366561837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 msgr2=0x562dc0ddf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0ddf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 -- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 msgr2=0x562dc0ddae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0ddae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 -- 172.21.15.37:0/2366561837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc0ddf9a0 con 0x562dc0dbd6f0 2024-04-26T18:24:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f94537fe700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0ddae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9452ffd700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x562dc0dc7d40 tx=0x7f9448011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c79fb4426d70e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.061+0000 7f9432ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448021060 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944402f070 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f9448003680 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9448004520 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 10 client.15342 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562dc0e3df20 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944402f070 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f944801b030 con 0x562dc0dbd6f0 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 10 client.15342 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:24:49.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 20 client.15342 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0e42b30 0x562dc0e41840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 4 client.15342 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.062+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 --> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] -- command(tid 0: {"prefix": "status"}) -- 0x562dc0d17260 con 0x562dc0e42b30 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f94537fe700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0e42b30 0x562dc0e41840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f9432ffd700 10 client.15342 ms_handle_connect on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f94537fe700 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0e42b30 0x562dc0e41840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10a2ba1994452a96 server_cookie=2ebb5a6f616fe5f7 in_seq=0 out_seq=0 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f9432ffd700 1 -- 172.21.15.37:0/2366561837 <== mds.1 v2:172.21.15.120:6836/1266100832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562dc0d17260 con 0x562dc0e42b30 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f9432ffd700 10 client.15342 handle_command_reply: tid=0 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.063+0000 7f94649d8b80 1 client.15342 shutdown 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 msgr2=0x562dc0ddb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x562dc0dc7d40 tx=0x7f9448011aa0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 shutdown_connections 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:6836/1266100832,v1:172.21.15.120:6837/1266100832] conn(0x562dc0e42b30 0x562dc0e41840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562dc0dc2a20 0x562dc0ddf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562dc0dbcd50 0x562dc0ddae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 --2- 172.21.15.37:0/2366561837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dc0dbd6f0 0x562dc0ddb3c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 >> 172.21.15.37:0/2366561837 conn(0x562dc0ceb570 msgr2=0x562dc0db7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f9432ffd700 0 client.15342 ms_handle_reset on v2:172.21.15.120:6836/1266100832 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 shutdown_connections 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 1 -- 172.21.15.37:0/2366561837 wait complete. 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f94649d8b80 20 client.15342 trim_cache size 0 max 16384 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:49.069 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 205.133781992, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 93, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 246.76936111099999 2024-04-26T18:24:49.070 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 msgr2=0x7f943c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 0x7f943c0a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9444009a10 tx=0x7f9444009190 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 msgr2=0x7f9454104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f944c00b830 tx=0x7f944c004ce0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 shutdown_connections 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9454108390 0x7f9454104a80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9454102320 0x7f9454104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f943c0a3ed0 0x7f943c0a6390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 --2- 172.21.15.37:0/4103468049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9454102ce0 0x7f9454104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:49.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.064+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 >> 172.21.15.37:0/4103468049 conn(0x7f94540792d0 msgr2=0x7f94541015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:49.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.065+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 shutdown_connections 2024-04-26T18:24:49.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:49.065+0000 7f945a146700 1 -- 172.21.15.37:0/4103468049 wait complete. 2024-04-26T18:24:49.421 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:49 smithi037 ceph-mon[29171]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:24:49.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:49 smithi120 ceph-mon[32244]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:24:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:49 smithi170 ceph-mon[32697]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:24:49.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:24:50.050 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:51.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:51 smithi037 ceph-mon[29171]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-26T18:24:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:51 smithi120 ceph-mon[32244]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-26T18:24:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:51 smithi170 ceph-mon[32697]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 -- 172.21.15.37:0/1703472354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e74107a90 msgr2=0x7f0e74107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 --2- 172.21.15.37:0/1703472354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e74107a90 0x7f0e74107e70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009990 tx=0x7f0e6c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 -- 172.21.15.37:0/1703472354 shutdown_connections 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 --2- 172.21.15.37:0/1703472354 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e741024a0 0x7f0e7410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 --2- 172.21.15.37:0/1703472354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e74107a90 0x7f0e74107e70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 --2- 172.21.15.37:0/1703472354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74101ae0 0x7f0e74101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 -- 172.21.15.37:0/1703472354 >> 172.21.15.37:0/1703472354 conn(0x7f0e74078990 msgr2=0x7f0e740ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 -- 172.21.15.37:0/1703472354 shutdown_connections 2024-04-26T18:24:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.699+0000 7f0e7976e700 1 -- 172.21.15.37:0/1703472354 wait complete. 2024-04-26T18:24:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 Processor -- start 2024-04-26T18:24:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 -- start start 2024-04-26T18:24:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 0x7f0e7419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74107a90 0x7f0e741a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e74111d30 con 0x7f0e741024a0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e74111bb0 con 0x7f0e74101ae0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e7976e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0e74111eb0 con 0x7f0e74107a90 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e727fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.700+0000 7f0e727fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45652/0 (socket says 172.21.15.37:45652) 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 -- 172.21.15.37:0/2970594924 learned_addr learned my addr 172.21.15.37:0/2970594924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 0x7f0e7419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e737fe700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74107a90 0x7f0e741a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 -- 172.21.15.37:0/2970594924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74107a90 msgr2=0x7f0e741a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74107a90 0x7f0e741a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 -- 172.21.15.37:0/2970594924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 msgr2=0x7f0e7419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 0x7f0e7419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 -- 172.21.15.37:0/2970594924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e741a6070 con 0x7f0e741024a0 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 0x7f0e7419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00b530 tx=0x7f0e5c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e6bfff700 1 -- 172.21.15.37:0/2970594924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c015020 con 0x7f0e741024a0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e6bfff700 1 -- 172.21.15.37:0/2970594924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c011ce0 con 0x7f0e741024a0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e741a62a0 con 0x7f0e741024a0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e6bfff700 1 -- 172.21.15.37:0/2970594924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c010e40 con 0x7f0e741024a0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.701+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e741a67b0 con 0x7f0e741024a0 2024-04-26T18:24:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.704+0000 7f0e6bfff700 1 -- 172.21.15.37:0/2970594924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0e5c018020 con 0x7f0e741024a0 2024-04-26T18:24:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e6bfff700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 0x7f0e600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 0x7f0e600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e6bfff700 1 -- 172.21.15.37:0/2970594924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0e5c0d01a0 con 0x7f0e741024a0 2024-04-26T18:24:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 0x7f0e600a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006010 tx=0x7f0e6c01a040 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e84000b80 1 Processor -- start 2024-04-26T18:24:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.705+0000 7f0e84000b80 1 -- start start 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858cd50 0x56345858d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x563458599770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5634584df7e0 con 0x563458592a20 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634584458e0 con 0x56345858d6f0 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e84000b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5634584db7a0 con 0x56345858cd50 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45656/0 (socket says 172.21.15.37:45656) 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 -- 172.21.15.37:0/3844872035 learned_addr learned my addr 172.21.15.37:0/3844872035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:51.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e727fc700 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858cd50 0x56345858d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e737fe700 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x563458599770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 -- 172.21.15.37:0/3844872035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858cd50 msgr2=0x56345858d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858cd50 0x56345858d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 -- 172.21.15.37:0/3844872035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 msgr2=0x563458599770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x563458599770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.706+0000 7f0e72ffd700 1 -- 172.21.15.37:0/3844872035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56345842c880 con 0x563458592a20 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bc60 tx=0x7f0e6c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1736d6fac5dcef05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e697fa700 1 -- 172.21.15.37:0/3844872035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00fa70 con 0x563458592a20 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e697fa700 1 -- 172.21.15.37:0/3844872035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c032070 con 0x563458592a20 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e697fa700 1 -- 172.21.15.37:0/3844872035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c023df0 con 0x563458592a20 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e84000b80 1 -- 172.21.15.37:0/3844872035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 msgr2=0x563458592e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.707+0000 7f0e84000b80 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bc60 tx=0x7f0e6c01e000 comp rx=0 tx=0).stop 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 -- 172.21.15.37:0/3844872035 shutdown_connections 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x563458599770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x563458592e00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 --2- 172.21.15.37:0/3844872035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858cd50 0x56345858d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 -- 172.21.15.37:0/3844872035 >> 172.21.15.37:0/3844872035 conn(0x5634584bb570 msgr2=0x563458587900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 -- 172.21.15.37:0/3844872035 shutdown_connections 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 -- 172.21.15.37:0/3844872035 wait complete. 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 -- start start 2024-04-26T18:24:51.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 Processor -- start 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.708+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x5634585966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563458592a20 0x563458596c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563458599cb0 con 0x56345858cd50 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563458499a30 con 0x56345858d6f0 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563458599e30 con 0x563458592a20 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 10 client.0 _ll_register_callbacks cb 0x563458477900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45666/0 (socket says 172.21.15.37:45666) 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 -- 172.21.15.37:0/641553546 learned_addr learned my addr 172.21.15.37:0/641553546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:51.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e727fc700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x5634585966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e737fe700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563458592a20 0x563458596c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 -- 172.21.15.37:0/641553546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563458592a20 msgr2=0x563458596c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563458592a20 0x563458596c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 -- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 msgr2=0x5634585966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x5634585966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 -- 172.21.15.37:0/641553546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56345849d430 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e727fc700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x5634585966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009350 tx=0x7f0e6c023740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b40761bdd3a954 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e5a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00fd40 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0e6c036420 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.709+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c032070 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c048050 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 10 client.15354 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5634585a43f0 con 0x56345858cd50 2024-04-26T18:24:51.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c032070 con 0x56345858cd50 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0e6c02d510 con 0x56345858cd50 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 10 client.15354 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 20 client.15354 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5634585a5740 0x5634585a7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 4 client.15354 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.710+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634584ed1c0 con 0x5634585a5740 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.711+0000 7f0e727fc700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5634585a5740 0x5634585a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.711+0000 7f0e727fc700 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5634585a5740 0x5634585a7900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faefdde9c7e8b72d server_cookie=b15b60d486f97e0a in_seq=0 out_seq=0 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.711+0000 7f0e5a7fc700 10 client.15354 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/641553546 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634584ed1c0 con 0x5634585a5740 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e5a7fc700 10 client.15354 handle_command_reply: tid=0 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 client.15354 shutdown 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 msgr2=0x563458599340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009350 tx=0x7f0e6c023740 comp rx=0 tx=0).stop 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 shutdown_connections 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5634585a5740 0x5634585a7900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563458592a20 0x563458596c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x563458599340 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 --2- 172.21.15.37:0/641553546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858d6f0 0x5634585966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e5a7fc700 0 client.15354 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 >> 172.21.15.37:0/641553546 conn(0x5634584bb570 msgr2=0x56345858bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 shutdown_connections 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 1 -- 172.21.15.37:0/641553546 wait complete. 2024-04-26T18:24:51.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.712+0000 7f0e84000b80 20 client.15354 trim_cache size 0 max 16384 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 Processor -- start 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 -- start start 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858cd50 0x563458597280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x56345858c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5634585a5c40 con 0x563458592a20 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634585a6310 con 0x56345858cd50 2024-04-26T18:24:51.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e84000b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5634585a69e0 con 0x56345858d6f0 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45680/0 (socket says 172.21.15.37:45680) 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 -- 172.21.15.37:0/541641690 learned_addr learned my addr 172.21.15.37:0/541641690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858cd50 0x563458597280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e727fc700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x56345858c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 -- 172.21.15.37:0/541641690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 msgr2=0x56345858c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x56345858c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 -- 172.21.15.37:0/541641690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858cd50 msgr2=0x563458597280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858cd50 0x563458597280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e737fe700 1 -- 172.21.15.37:0/541641690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634585a75a0 con 0x563458592a20 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.733+0000 7f0e727fc700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x56345858c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.734+0000 7f0e737fe700 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009f80 tx=0x7f0e64009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ddd3e0418d839d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.734+0000 7f0e68ff9700 1 -- 172.21.15.37:0/541641690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6401b020 con 0x563458592a20 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.734+0000 7f0e68ff9700 1 -- 172.21.15.37:0/541641690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6400ae50 con 0x563458592a20 2024-04-26T18:24:51.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.734+0000 7f0e68ff9700 1 -- 172.21.15.37:0/541641690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64016840 con 0x563458592a20 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- 172.21.15.37:0/541641690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 msgr2=0x56345858f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009f80 tx=0x7f0e64009650 comp rx=0 tx=0).stop 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- 172.21.15.37:0/541641690 shutdown_connections 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563458592a20 0x56345858f360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56345858cd50 0x563458597280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- 172.21.15.37:0/541641690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x56345858c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- 172.21.15.37:0/541641690 >> 172.21.15.37:0/541641690 conn(0x5634584bb570 msgr2=0x56345858bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- 172.21.15.37:0/541641690 shutdown_connections 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- 172.21.15.37:0/541641690 wait complete. 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- start start 2024-04-26T18:24:51.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 Processor -- start 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x5634585ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 0x5634585ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5634585a7780 con 0x56345858cd50 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5634585a70b0 con 0x563458592a20 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5634585a7e50 con 0x56345858d6f0 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.735+0000 7f0e84000b80 10 client.0 _ll_register_callbacks cb 0x563458477900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45694/0 (socket says 172.21.15.37:45694) 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 -- 172.21.15.37:0/2904756125 learned_addr learned my addr 172.21.15.37:0/2904756125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e737fe700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 0x5634585ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x5634585ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 -- 172.21.15.37:0/2904756125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 msgr2=0x5634585ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x5634585ac270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 -- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 msgr2=0x5634585ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 0x5634585ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 -- 172.21.15.37:0/2904756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56345860d5c0 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x5634585ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e737fe700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 0x5634585ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e72ffd700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009990 tx=0x7f0e6c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a65608cc407b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e5a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c02d5f0 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0e6c023860 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6400ae50 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c053050 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e84000b80 10 client.15360 fetch_fsmap learned FSMap version 26 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.736+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56345860e770 con 0x56345858cd50 2024-04-26T18:24:51.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6400ae50 con 0x56345858cd50 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0e6c024410 con 0x56345858cd50 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 10 client.15360 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 20 client.15360 populate_metadata read hostname 'smithi037' 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x563458613250 0x563458611f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 4 client.15360 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "status"}) -- 0x5634584e7260 con 0x563458613250 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.737+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x563458613250 0x563458611f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e5a7fc700 10 client.15360 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e727fc700 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x563458613250 0x563458611f60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd4d9bc5ef4731 server_cookie=12adf2316937ab9f in_seq=0 out_seq=0 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e5a7fc700 1 -- 172.21.15.37:0/2904756125 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5634584e7260 con 0x563458613250 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e5a7fc700 10 client.15360 handle_command_reply: tid=0 2024-04-26T18:24:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 client.15360 shutdown 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 msgr2=0x5634585abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009990 tx=0x7f0e6c00f7b0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 shutdown_connections 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x563458613250 0x563458611f60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563458592a20 0x5634585ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56345858cd50 0x5634585abd30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e5a7fc700 0 client.15360 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 --2- 172.21.15.37:0/2904756125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56345858d6f0 0x5634585ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 >> 172.21.15.37:0/2904756125 conn(0x5634584bb570 msgr2=0x56345858c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 shutdown_connections 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 1 -- 172.21.15.37:0/2904756125 wait complete. 2024-04-26T18:24:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.738+0000 7f0e84000b80 20 client.15360 trim_cache size 0 max 16384 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "755ef512-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24464, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 219.83331020700001, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 93, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 236.205239597 2024-04-26T18:24:51.744 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 msgr2=0x7f0e600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 0x7f0e600a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006010 tx=0x7f0e6c01a040 comp rx=0 tx=0).stop 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 msgr2=0x7f0e7419aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00b530 tx=0x7f0e5c00b840 comp rx=0 tx=0).stop 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 shutdown_connections 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e74107a90 0x7f0e741a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0e600a3ed0 0x7f0e600a6390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e74101ae0 0x7f0e7419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 --2- 172.21.15.37:0/2970594924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e741024a0 0x7f0e7419aff0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 >> 172.21.15.37:0/2970594924 conn(0x7f0e74078990 msgr2=0x7f0e740ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 shutdown_connections 2024-04-26T18:24:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:24:51.739+0000 7f0e7976e700 1 -- 172.21.15.37:0/2970594924 wait complete. 2024-04-26T18:24:52.437 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T18:24:53.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:53 smithi037 ceph-mon[29171]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:53 smithi120 ceph-mon[32244]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:53 smithi170 ceph-mon[32697]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:55.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:55 smithi037 ceph-mon[29171]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T18:24:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:55 smithi120 ceph-mon[32244]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T18:24:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:55 smithi170 ceph-mon[32697]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-26T18:24:57.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:57 smithi037 ceph-mon[29171]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:57 smithi120 ceph-mon[32244]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:57 smithi170 ceph-mon[32697]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-04-26T18:24:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:24:59 smithi037 ceph-mon[29171]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:24:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:24:59 smithi120 ceph-mon[32244]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:24:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:24:59 smithi170 ceph-mon[32697]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T18:25:00.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:25:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:25:00.438+0000 7fab19737700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:25:00.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:25:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:25:00.438+0000 7fab19737700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:25:00.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:25:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:25:00.438+0000 7fab19737700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:25:00.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:25:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:25:00.438+0000 7fab19737700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:25:00.733 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:25:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:25:00.438+0000 7fab19737700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:25:01.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:01 smithi037 ceph-mon[29171]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T18:25:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:01 smithi120 ceph-mon[32244]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T18:25:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:01 smithi170 ceph-mon[32697]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T18:25:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 -- 172.21.15.37:0/3526955875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4104500 msgr2=0x7f88f410c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 --2- 172.21.15.37:0/3526955875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4104500 0x7f88f410c9e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009900 tx=0x7f88e802a480 comp rx=0 tx=0).stop 2024-04-26T18:25:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 -- 172.21.15.37:0/3526955875 shutdown_connections 2024-04-26T18:25:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 --2- 172.21.15.37:0/3526955875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f410cfb0 0x7f88f410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 --2- 172.21.15.37:0/3526955875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f4104500 0x7f88f410c9e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 --2- 172.21.15.37:0/3526955875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 0x7f88f4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 -- 172.21.15.37:0/3526955875 >> 172.21.15.37:0/3526955875 conn(0x7f88f4077bb0 msgr2=0x7f88f4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 -- 172.21.15.37:0/3526955875 shutdown_connections 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 -- 172.21.15.37:0/3526955875 wait complete. 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.701+0000 7f88f9e07640 1 Processor -- start 2024-04-26T18:25:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 -- start start 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 0x7f88f41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f4104500 0x7f88f41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4112bf0 con 0x7f88f410cfb0 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4112a70 con 0x7f88f4103be0 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f9e07640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4112d70 con 0x7f88f4104500 2024-04-26T18:25:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34796/0 (socket says 172.21.15.37:34796) 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 -- 172.21.15.37:0/1687556416 learned_addr learned my addr 172.21.15.37:0/1687556416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f4104500 0x7f88f41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f37fe640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 0x7f88f41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 -- 172.21.15.37:0/1687556416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f4104500 msgr2=0x7f88f41995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f4104500 0x7f88f41995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 -- 172.21.15.37:0/1687556416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 msgr2=0x7f88f41990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 0x7f88f41990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.702+0000 7f88f3fff640 1 -- 172.21.15.37:0/1687556416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f419e050 con 0x7f88f410cfb0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f3fff640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00dac0 tx=0x7f88ec00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f0ff9640 1 -- 172.21.15.37:0/1687556416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec017070 con 0x7f88f410cfb0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f419e340 con 0x7f88f410cfb0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f0ff9640 1 -- 172.21.15.37:0/1687556416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ec00be30 con 0x7f88f410cfb0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f0ff9640 1 -- 172.21.15.37:0/1687556416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec010460 con 0x7f88f410cfb0 2024-04-26T18:25:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.703+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41a5bd0 con 0x7f88f410cfb0 2024-04-26T18:25:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.705+0000 7f88f0ff9640 1 -- 172.21.15.37:0/1687556416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f88ec0040b0 con 0x7f88f410cfb0 2024-04-26T18:25:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.705+0000 7f88fcbfa740 1 Processor -- start 2024-04-26T18:25:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.705+0000 7f88fcbfa740 1 -- start start 2024-04-26T18:25:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x558755840ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587558410e0 0x5587558434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55875579c460 con 0x558755835f00 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558755756590 con 0x558755836820 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558755798420 con 0x5587558410e0 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34808/0 (socket says 172.21.15.37:34808) 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 -- 172.21.15.37:0/229451898 learned_addr learned my addr 172.21.15.37:0/229451898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f37fe640 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x558755840ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f0ff9640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 0x7f88dc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 -- 172.21.15.37:0/229451898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587558410e0 msgr2=0x5587558434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587558410e0 0x5587558434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f3fff640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 0x7f88dc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 -- 172.21.15.37:0/229451898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 msgr2=0x558755840ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x558755840ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 -- 172.21.15.37:0/229451898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587557630d0 con 0x558755835f00 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f0ff9640 1 -- 172.21.15.37:0/1687556416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f88ec0d0660 con 0x7f88f410cfb0 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.706+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f88e802a9a0 tx=0x7f88e80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f44c008982533dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88d9ffb640 1 -- 172.21.15.37:0/229451898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8035040 con 0x558755835f00 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88f3fff640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 0x7f88dc0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0cdc80 tx=0x7f88ec06bd30 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88d9ffb640 1 -- 172.21.15.37:0/229451898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e80316a0 con 0x558755835f00 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88d9ffb640 1 -- 172.21.15.37:0/229451898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8004750 con 0x558755835f00 2024-04-26T18:25:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 -- 172.21.15.37:0/229451898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 msgr2=0x5587558362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f88e802a9a0 tx=0x7f88e80095a0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 -- 172.21.15.37:0/229451898 shutdown_connections 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x558755840ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587558410e0 0x5587558434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/229451898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755835f00 0x5587558362e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 -- 172.21.15.37:0/229451898 >> 172.21.15.37:0/229451898 conn(0x558755832410 msgr2=0x5587558327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 -- 172.21.15.37:0/229451898 shutdown_connections 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.707+0000 7f88fcbfa740 1 -- 172.21.15.37:0/229451898 wait complete. 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 -- start start 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 Processor -- start 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558755843cb0 con 0x5587558410e0 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558755723b60 con 0x558755836820 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558755843e30 con 0x558755835f00 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88fcbfa740 10 client.0 _ll_register_callbacks cb 0x5587557482b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.708+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34812/0 (socket says 172.21.15.37:34812) 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 -- 172.21.15.37:0/3828791484 learned_addr learned my addr 172.21.15.37:0/3828791484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 -- 172.21.15.37:0/3828791484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 msgr2=0x55875583e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 -- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 msgr2=0x55875583e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 -- 172.21.15.37:0/3828791484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587554ae210 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f88e8005180 tx=0x7f88e8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d941e6cb89045e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8035070 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f88e8031b10 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e80316a0 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88e8058050 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88fcbfa740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.709+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55875557a620 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e80316a0 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f88e8004960 con 0x5587558410e0 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 20 client.15378 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558755851070 0x558755853460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 4 client.15378 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587557aa070 con 0x558755851070 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.710+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558755851070 0x558755853460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.711+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558755851070 0x558755853460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfc5d4ee4b4c1ff server_cookie=76d05eeb78cfb4b1 in_seq=0 out_seq=0 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.711+0000 7f88d2ffd640 10 client.15378 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.711+0000 7f88d2ffd640 1 -- 172.21.15.37:0/3828791484 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587557aa070 con 0x558755851070 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.711+0000 7f88d2ffd640 10 client.15378 handle_command_reply: tid=0 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.711+0000 7f88fcbfa740 1 client.15378 shutdown 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 msgr2=0x55875583eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f88e8005180 tx=0x7f88e8005250 comp rx=0 tx=0).stop 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 shutdown_connections 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558755851070 0x558755853460 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583eb10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88d2ffd640 0 client.15378 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3828791484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 >> 172.21.15.37:0/3828791484 conn(0x558755832410 msgr2=0x558755838190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 shutdown_connections 2024-04-26T18:25:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3828791484 wait complete. 2024-04-26T18:25:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.712+0000 7f88fcbfa740 20 client.15378 trim_cache size 0 max 16384 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 Processor -- start 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 -- start start 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875584cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558410e0 0x558755858a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41b0cf0 con 0x558755836820 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41ae5c0 con 0x5587558410e0 2024-04-26T18:25:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41ac410 con 0x558755835f00 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34820/0 (socket says 172.21.15.37:34820) 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88f37fe640 1 -- 172.21.15.37:0/3379625725 learned_addr learned my addr 172.21.15.37:0/3379625725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.727+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558410e0 0x558755858a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875584cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 -- 172.21.15.37:0/3379625725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 msgr2=0x55875584cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875584cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 -- 172.21.15.37:0/3379625725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558410e0 msgr2=0x558755858a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558410e0 0x558755858a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 -- 172.21.15.37:0/3379625725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558755843700 con 0x558755836820 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875584cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88f37fe640 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x558755853a30 tx=0x7f88e400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616eeaa9ec263dae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88d97fa640 1 -- 172.21.15.37:0/3379625725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4003ea0 con 0x558755836820 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88d97fa640 1 -- 172.21.15.37:0/3379625725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e401f5a0 con 0x558755836820 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88d97fa640 1 -- 172.21.15.37:0/3379625725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4011a70 con 0x558755836820 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3379625725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 msgr2=0x55875583d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.728+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x558755853a30 tx=0x7f88e400b390 comp rx=0 tx=0).stop 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3379625725 shutdown_connections 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587558410e0 0x558755858a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558755836820 0x55875583d250 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/3379625725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875584cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3379625725 >> 172.21.15.37:0/3379625725 conn(0x558755832410 msgr2=0x55875582d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3379625725 shutdown_connections 2024-04-26T18:25:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- 172.21.15.37:0/3379625725 wait complete. 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- start start 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 Processor -- start 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41aca80 con 0x5587558410e0 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41adef0 con 0x558755836820 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.729+0000 7f88fcbfa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41ad150 con 0x558755835f00 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88fcbfa740 10 client.0 _ll_register_callbacks cb 0x7f88e400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34824/0 (socket says 172.21.15.37:34824) 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f2ffd640 1 -- 172.21.15.37:0/1176843607 learned_addr learned my addr 172.21.15.37:0/1176843607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 -- 172.21.15.37:0/1176843607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 msgr2=0x55875583aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 -- 172.21.15.37:0/1176843607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 msgr2=0x55875583b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 -- 172.21.15.37:0/1176843607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587558b6460 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88f37fe640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583b430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5587558358b0 tx=0x7f88e4011670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77db496e5bf888a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e401f2f0 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e401f5a0 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.730+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f88e4003ca0 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88e401f910 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 10 client.25046 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587558b76c0 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e401f5a0 con 0x558755836820 2024-04-26T18:25:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f88e4030540 con 0x558755836820 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 10 client.25046 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 10 client.25046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 20 client.25046 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587558b7dc0 0x5587558ba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 4 client.25046 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587557a3ee0 con 0x5587558b7dc0 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.731+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587558b7dc0 0x5587558ba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.732+0000 7f88f2ffd640 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587558b7dc0 0x5587558ba1e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c6149535ca84ed server_cookie=ac512544d589d2e4 in_seq=0 out_seq=0 2024-04-26T18:25:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.732+0000 7f88d2ffd640 10 client.25046 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.739+0000 7f88d2ffd640 1 -- 172.21.15.37:0/1176843607 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587557a3ee0 con 0x5587558b7dc0 2024-04-26T18:25:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.739+0000 7f88d2ffd640 10 client.25046 handle_command_reply: tid=0 2024-04-26T18:25:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.739+0000 7f88fcbfa740 1 client.25046 shutdown 2024-04-26T18:25:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 msgr2=0x55875583b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583b430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5587558358b0 tx=0x7f88e4011670 comp rx=0 tx=0).stop 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 shutdown_connections 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587558b7dc0 0x5587558ba1e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558755836820 0x55875583b430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587558410e0 0x55875583b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 --2- 172.21.15.37:0/1176843607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558755835f00 0x55875583aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88d2ffd640 0 client.25046 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 >> 172.21.15.37:0/1176843607 conn(0x558755832410 msgr2=0x55875583f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 shutdown_connections 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 1 -- 172.21.15.37:0/1176843607 wait complete. 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88fcbfa740 20 client.25046 trim_cache size 0 max 16384 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:25:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:25:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:25:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 msgr2=0x7f88dc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 0x7f88dc0a63a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0cdc80 tx=0x7f88ec06bd30 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 msgr2=0x7f88f419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00dac0 tx=0x7f88ec00df80 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 shutdown_connections 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f88dc0a3ee0 0x7f88dc0a63a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88f410cfb0 0x7f88f419d920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88f4104500 0x7f88f41995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 --2- 172.21.15.37:0/1687556416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88f4103be0 0x7f88f41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.740+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 >> 172.21.15.37:0/1687556416 conn(0x7f88f4077bb0 msgr2=0x7f88f40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.741+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 shutdown_connections 2024-04-26T18:25:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:01.741+0000 7f88f9e07640 1 -- 172.21.15.37:0/1687556416 wait complete. 2024-04-26T18:25:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:03 smithi037 ceph-mon[29171]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T18:25:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:03 smithi037 ceph-mon[29171]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-26T18:25:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:03 smithi120 ceph-mon[32244]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T18:25:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:03 smithi120 ceph-mon[32244]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-26T18:25:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:03 smithi170 ceph-mon[32697]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T18:25:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:03 smithi170 ceph-mon[32697]: mgrmap e25: x(active, since 42s), standbys: z, y 2024-04-26T18:25:05.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:05 smithi037 ceph-mon[29171]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:25:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:05 smithi120 ceph-mon[32244]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:25:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:05 smithi170 ceph-mon[32697]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:25:07.437 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:25:07.438 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:25:07.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:25:07.469 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:07 smithi037 ceph-mon[29171]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:07 smithi120 ceph-mon[32244]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:07 smithi170 ceph-mon[32697]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:07.705 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:09.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 -- 172.21.15.37:0/2986126402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 msgr2=0x7fae60102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 --2- 172.21.15.37:0/2986126402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60102990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fae480099c0 tx=0x7fae48009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 -- 172.21.15.37:0/2986126402 shutdown_connections 2024-04-26T18:25:09.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 --2- 172.21.15.37:0/2986126402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 0x7fae6010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 --2- 172.21.15.37:0/2986126402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60102990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 --2- 172.21.15.37:0/2986126402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 0x7fae60106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 -- 172.21.15.37:0/2986126402 >> 172.21.15.37:0/2986126402 conn(0x7fae600792d0 msgr2=0x7fae600796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 -- 172.21.15.37:0/2986126402 shutdown_connections 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.445+0000 7fae64d05700 1 -- 172.21.15.37:0/2986126402 wait complete. 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 Processor -- start 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 -- start start 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 0x7fae60198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 0x7fae60199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae601124d0 con 0x7fae601025b0 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae60112350 con 0x7fae60107640 2024-04-26T18:25:09.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae64d05700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae60112650 con 0x7fae60102f60 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53460/0 (socket says 172.21.15.37:53460) 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5e59c700 1 -- 172.21.15.37:0/114810351 learned_addr learned my addr 172.21.15.37:0/114810351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5ed9d700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 0x7fae60199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5dd9b700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 0x7fae60198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.446+0000 7fae5e59c700 1 -- 172.21.15.37:0/114810351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 msgr2=0x7fae60198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae5e59c700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 0x7fae60198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae5e59c700 1 -- 172.21.15.37:0/114810351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 msgr2=0x7fae60199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae5e59c700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 0x7fae60199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae5e59c700 1 -- 172.21.15.37:0/114810351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae601a7f70 con 0x7fae601025b0 2024-04-26T18:25:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae5e59c700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fae480098e0 tx=0x7fae480057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48027070 con 0x7fae601025b0 2024-04-26T18:25:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae601a8200 con 0x7fae601025b0 2024-04-26T18:25:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae48003f20 con 0x7fae601025b0 2024-04-26T18:25:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48022ca0 con 0x7fae601025b0 2024-04-26T18:25:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.447+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601a8650 con 0x7fae601025b0 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.448+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6006a110 con 0x7fae601025b0 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.449+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fae4802a070 con 0x7fae601025b0 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.450+0000 7fae577fe700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 0x7fae4c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.450+0000 7fae5dd9b700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 0x7fae4c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.450+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fae480d1950 con 0x7fae601025b0 2024-04-26T18:25:09.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.450+0000 7fae5dd9b700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 0x7fae4c0a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae6019a860 tx=0x7fae5000a380 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:09.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.454+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae4805eff0 con 0x7fae601025b0 2024-04-26T18:25:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:09 smithi037 ceph-mon[29171]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:09 smithi120 ceph-mon[32244]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:09 smithi170 ceph-mon[32697]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.714+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae6002e490 con 0x7fae601025b0 2024-04-26T18:25:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.715+0000 7fae577fe700 1 -- 172.21.15.37:0/114810351 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7fae48013070 con 0x7fae601025b0 2024-04-26T18:25:09.715 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:25:09.715 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:25:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 msgr2=0x7fae4c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 0x7fae4c0a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fae6019a860 tx=0x7fae5000a380 comp rx=0 tx=0).stop 2024-04-26T18:25:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 msgr2=0x7fae60198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fae480098e0 tx=0x7fae480057c0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 shutdown_connections 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fae4c0a3f60 0x7fae4c0a6420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae60107640 0x7fae60199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae601025b0 0x7fae60198700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 --2- 172.21.15.37:0/114810351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae60102f60 0x7fae60198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 >> 172.21.15.37:0/114810351 conn(0x7fae600792d0 msgr2=0x7fae60101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 shutdown_connections 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:09.717+0000 7fae64d05700 1 -- 172.21.15.37:0/114810351 wait complete. 2024-04-26T18:25:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:25:10.412 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:10 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/114810351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:10.442 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:25:10.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:25:10.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:10 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/114810351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:10.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:10 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/114810351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:11 smithi037 ceph-mon[29171]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:11 smithi120 ceph-mon[32244]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:11 smithi170 ceph-mon[32697]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2089367036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0107650 msgr2=0x7f3cd010f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2089367036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0107650 0x7f3cd010f220 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80099c0 tx=0x7f3cc8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2089367036 shutdown_connections 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2089367036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0107650 0x7f3cd010f220 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2089367036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102580 0x7f3cd0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2089367036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0102f30 0x7f3cd0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2089367036 >> 172.21.15.37:0/2089367036 conn(0x7f3cd00792c0 msgr2=0x7f3cd00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2089367036 shutdown_connections 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.482+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2089367036 wait complete. 2024-04-26T18:25:12.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 Processor -- start 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 -- start start 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102f30 0x7f3cd019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0107650 0x7f3cd01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01124f0 con 0x7f3cd0102580 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0112370 con 0x7f3cd0107650 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd7b2c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0112670 con 0x7f3cd0102f30 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.483+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53478/0 (socket says 172.21.15.37:53478) 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2477435113 learned_addr learned my addr 172.21.15.37:0/2477435113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102f30 0x7f3cd019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0107650 0x7f3cd01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2477435113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102f30 msgr2=0x7f3cd019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102f30 0x7f3cd019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2477435113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0107650 msgr2=0x7f3cd01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0107650 0x7f3cd01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2477435113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd0199f80 con 0x7f3cd0102580 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000ada0 tx=0x7f3cc000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cceffd700 1 -- 172.21.15.37:0/2477435113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0017020 con 0x7f3cd0102580 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cceffd700 1 -- 172.21.15.37:0/2477435113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc000ce30 con 0x7f3cd0102580 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd019a270 con 0x7f3cd0102580 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cceffd700 1 -- 172.21.15.37:0/2477435113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0005480 con 0x7f3cd0102580 2024-04-26T18:25:12.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.484+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01a6480 con 0x7f3cd0102580 2024-04-26T18:25:12.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.487+0000 7f3cceffd700 1 -- 172.21.15.37:0/2477435113 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f3cc0004ad0 con 0x7f3cd0102580 2024-04-26T18:25:12.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.488+0000 7f3cceffd700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 0x7f3cb80a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.488+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 0x7f3cb80a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.488+0000 7f3cceffd700 1 -- 172.21.15.37:0/2477435113 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3cc00d0300 con 0x7f3cd0102580 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.488+0000 7f3ce23beb80 1 Processor -- start 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.488+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 0x7f3cb80a64c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cd01088d0 tx=0x7f3cc400a380 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 -- start start 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d12610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d24200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b093c658d0 con 0x55b093d12b50 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b093c04770 con 0x55b093d21e10 2024-04-26T18:25:12.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b093c61890 con 0x55b093d12230 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53480/0 (socket says 172.21.15.37:53480) 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 -- 172.21.15.37:0/3956158801 learned_addr learned my addr 172.21.15.37:0/3956158801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d24200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d12610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 -- 172.21.15.37:0/3956158801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 msgr2=0x55b093d12610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d12610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 -- 172.21.15.37:0/3956158801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 msgr2=0x55b093d24200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d24200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd50c7700 1 -- 172.21.15.37:0/3956158801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b093bb2a60 con 0x55b093d12b50 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.489+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d12610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4004500 tx=0x7f3cc4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f3e455285ad808 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3cb7fff700 1 -- 172.21.15.37:0/3956158801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4016070 con 0x55b093d12b50 2024-04-26T18:25:12.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3cb7fff700 1 -- 172.21.15.37:0/3956158801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc4014070 con 0x55b093d12b50 2024-04-26T18:25:12.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3cb7fff700 1 -- 172.21.15.37:0/3956158801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4010d40 con 0x55b093d12b50 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3ce23beb80 1 -- 172.21.15.37:0/3956158801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 msgr2=0x55b093d218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.490+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4004500 tx=0x7f3cc4012000 comp rx=0 tx=0).stop 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 -- 172.21.15.37:0/3956158801 shutdown_connections 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d24200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d12610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/3956158801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d218d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 -- 172.21.15.37:0/3956158801 >> 172.21.15.37:0/3956158801 conn(0x55b093c3fed0 msgr2=0x55b093d0e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 -- 172.21.15.37:0/3956158801 shutdown_connections 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 -- 172.21.15.37:0/3956158801 wait complete. 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 -- start start 2024-04-26T18:25:12.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.491+0000 7f3ce23beb80 1 Processor -- start 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d1e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d1ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b093d249e0 con 0x55b093d12230 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b093bcbae0 con 0x55b093d21e10 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b093d24b60 con 0x55b093d12b50 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3ce23beb80 10 client.0 _ll_register_callbacks cb 0x55b093bf9710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53496/0 (socket says 172.21.15.37:53496) 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2236691788 learned_addr learned my addr 172.21.15.37:0/2236691788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d1ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d1e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2236691788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 msgr2=0x55b093d1e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d1e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 msgr2=0x55b093d1ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d1ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 -- 172.21.15.37:0/2236691788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b093d1f3c0 con 0x55b093d12230 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d1e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0098ab0 tx=0x7f3cc009bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308fdb4a26ccf680 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00b5060 con 0x55b093d12230 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.492+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3cc00a0170 con 0x55b093d12230 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc4014070 con 0x55b093d12230 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cc00a0490 con 0x55b093d12230 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 10 client.15402 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b093d2f3e0 con 0x55b093d12230 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc4014070 con 0x55b093d12230 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3cc0012070 con 0x55b093d12230 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 10 client.15402 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 20 client.15402 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d31970 0x55b093d33d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 4 client.15402 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.493+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b093c732b0 con 0x55b093d31970 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.494+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d31970 0x55b093d33d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.494+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d31970 0x55b093d33d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcdb37c00511550 server_cookie=df38c7861486f5d7 in_seq=0 out_seq=0 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.494+0000 7f3cb57fa700 10 client.15402 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3cb57fa700 1 -- 172.21.15.37:0/2236691788 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b093c732b0 con 0x55b093d31970 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3cb57fa700 10 client.15402 handle_command_reply: tid=0 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 client.15402 shutdown 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 msgr2=0x55b093d21540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0098ab0 tx=0x7f3cc009bd50 comp rx=0 tx=0).stop 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 shutdown_connections 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d31970 0x55b093d33d60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d1ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d21540 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3cb57fa700 0 client.15402 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/2236691788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d1e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 >> 172.21.15.37:0/2236691788 conn(0x55b093c3fed0 msgr2=0x55b093d0e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 shutdown_connections 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 1 -- 172.21.15.37:0/2236691788 wait complete. 2024-04-26T18:25:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.495+0000 7f3ce23beb80 20 client.15402 trim_cache size 0 max 16384 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 Processor -- start 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 -- start start 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01ae3d0 con 0x55b093d12b50 2024-04-26T18:25:12.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01abc60 con 0x55b093d21e10 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01a9ab0 con 0x55b093d12230 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3cd50c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3cd50c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53504/0 (socket says 172.21.15.37:53504) 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3cd50c7700 1 -- 172.21.15.37:0/1313216994 learned_addr learned my addr 172.21.15.37:0/1313216994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.515+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 -- 172.21.15.37:0/1313216994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 msgr2=0x55b093d149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 -- 172.21.15.37:0/1313216994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 msgr2=0x55b093d395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 -- 172.21.15.37:0/1313216994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b093d0f1f0 con 0x55b093d12b50 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40044d0 tx=0x7f3cc400f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d89f9d72520e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cb77fe700 1 -- 172.21.15.37:0/1313216994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4016040 con 0x55b093d12b50 2024-04-26T18:25:12.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cb77fe700 1 -- 172.21.15.37:0/1313216994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc4014030 con 0x55b093d12b50 2024-04-26T18:25:12.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.516+0000 7f3cb77fe700 1 -- 172.21.15.37:0/1313216994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4010d30 con 0x55b093d12b50 2024-04-26T18:25:12.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1313216994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 msgr2=0x55b093d21080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40044d0 tx=0x7f3cc400f7d0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1313216994 shutdown_connections 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12230 0x55b093d149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1313216994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12b50 0x55b093d21080 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1313216994 >> 172.21.15.37:0/1313216994 conn(0x55b093c3fed0 msgr2=0x55b093d19ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1313216994 shutdown_connections 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1313216994 wait complete. 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 -- start start 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.517+0000 7f3ce23beb80 1 Processor -- start 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d2a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d26f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01aa120 con 0x55b093d12230 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01ab590 con 0x55b093d21e10 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01aa7f0 con 0x55b093d12b50 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3ce23beb80 10 client.0 _ll_register_callbacks cb 0x55b093bf9710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53518/0 (socket says 172.21.15.37:53518) 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 -- 172.21.15.37:0/1136626987 learned_addr learned my addr 172.21.15.37:0/1136626987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d2a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d26f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 -- 172.21.15.37:0/1136626987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 msgr2=0x55b093d2a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d2a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 -- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 msgr2=0x55b093d26f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d26f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 -- 172.21.15.37:0/1136626987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b093d277b0 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd60c9700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d26f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cd58c8700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000af80 tx=0x7f3cc0007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b06114e6abe219 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.518+0000 7f3cb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc000cc60 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3cc009b530 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc4014030 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cc00a4cc0 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 10 client.15408 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b093d98ae0 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc4014030 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3cc0012040 con 0x55b093d12230 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:12.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 10 client.15408 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 20 client.15408 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d991a0 0x55b093d9b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 4 client.15408 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.519+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55b093c6d350 con 0x55b093d991a0 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.520+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d991a0 0x55b093d9b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.520+0000 7f3cd50c7700 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d991a0 0x55b093d9b590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282934bb31e59b8c server_cookie=ba96457fd5036395 in_seq=0 out_seq=0 2024-04-26T18:25:12.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.520+0000 7f3cb57fa700 10 client.15408 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3cb57fa700 1 -- 172.21.15.37:0/1136626987 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55b093c6d350 con 0x55b093d991a0 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3cb57fa700 10 client.15408 handle_command_reply: tid=0 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 client.15408 shutdown 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 msgr2=0x55b093d29df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000af80 tx=0x7f3cc0007bf0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 shutdown_connections 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b093d991a0 0x55b093d9b590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b093d21e10 0x55b093d26f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b093d12230 0x55b093d29df0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3cb57fa700 0 client.15408 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 --2- 172.21.15.37:0/1136626987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b093d12b50 0x55b093d2a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 >> 172.21.15.37:0/1136626987 conn(0x55b093c3fed0 msgr2=0x55b093d149c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 shutdown_connections 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 1 -- 172.21.15.37:0/1136626987 wait complete. 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.534+0000 7f3ce23beb80 20 client.15408 trim_cache size 0 max 16384 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "b2739c93": { 2024-04-26T18:25:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:25:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 msgr2=0x7f3cb80a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 0x7f3cb80a64c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cd01088d0 tx=0x7f3cc400a380 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 msgr2=0x7f3cd019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000ada0 tx=0x7f3cc000c330 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 shutdown_connections 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3cb80a4000 0x7f3cb80a64c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cd0107650 0x7f3cd01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cd0102580 0x7f3cd019f620 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 --2- 172.21.15.37:0/2477435113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cd0102f30 0x7f3cd019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 >> 172.21.15.37:0/2477435113 conn(0x7f3cd00792c0 msgr2=0x7f3cd006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 shutdown_connections 2024-04-26T18:25:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:12.535+0000 7f3cd7b2c700 1 -- 172.21.15.37:0/2477435113 wait complete. 2024-04-26T18:25:13.344 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b2739c93', quiesce took 5.9 seconds. Will release after: 30.0 2024-04-26T18:25:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:13 smithi037 ceph-mon[29171]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:13 smithi120 ceph-mon[32244]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:13 smithi170 ceph-mon[32697]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T18:25:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 -- 172.21.15.37:0/3752507765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 msgr2=0x7f63801116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 --2- 172.21.15.37:0/3752507765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f63801116e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6378009ca0 tx=0x7f637802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 -- 172.21.15.37:0/3752507765 shutdown_connections 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 --2- 172.21.15.37:0/3752507765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380102060 0x7f638010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 --2- 172.21.15.37:0/3752507765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f63801116e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 --2- 172.21.15.37:0/3752507765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380101740 0x7f6380101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 -- 172.21.15.37:0/3752507765 >> 172.21.15.37:0/3752507765 conn(0x7f63800789b0 msgr2=0x7f6380078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 -- 172.21.15.37:0/3752507765 shutdown_connections 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.841+0000 7f6387469640 1 -- 172.21.15.37:0/3752507765 wait complete. 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 Processor -- start 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 -- start start 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 0x7f638019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 0x7f638019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6380114dd0 con 0x7f638010f2f0 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6380114c50 con 0x7f6380101740 2024-04-26T18:25:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6387469640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6380114f50 con 0x7f6380102060 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32888/0 (socket says 172.21.15.37:32888) 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385a06640 1 -- 172.21.15.37:0/1282891368 learned_addr learned my addr 172.21.15.37:0/1282891368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385205640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 0x7f638019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6384a04640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 0x7f638019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385a06640 1 -- 172.21.15.37:0/1282891368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 msgr2=0x7f638019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.842+0000 7f6385a06640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 0x7f638019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6385a06640 1 -- 172.21.15.37:0/1282891368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 msgr2=0x7f638019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6385a06640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 0x7f638019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6385a06640 1 -- 172.21.15.37:0/1282891368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63801a0250 con 0x7f638010f2f0 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6385205640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 0x7f638019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6384a04640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 0x7f638019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6385a06640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f63780319f0 tx=0x7f6378031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f63767fc640 1 -- 172.21.15.37:0/1282891368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378003bf0 con 0x7f638010f2f0 2024-04-26T18:25:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f63767fc640 1 -- 172.21.15.37:0/1282891368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6378003d90 con 0x7f638010f2f0 2024-04-26T18:25:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63801a04e0 con 0x7f638010f2f0 2024-04-26T18:25:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f63767fc640 1 -- 172.21.15.37:0/1282891368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637802fbc0 con 0x7f638010f2f0 2024-04-26T18:25:14.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.843+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63801a7dd0 con 0x7f638010f2f0 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.845+0000 7f63767fc640 1 -- 172.21.15.37:0/1282891368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f637802fd60 con 0x7f638010f2f0 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 Processor -- start 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 -- start start 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a16ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558646972460 con 0x558646a0bf00 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55864692c590 con 0x558646a170e0 2024-04-26T18:25:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f638a178740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55864696e420 con 0x558646a0c820 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32902/0 (socket says 172.21.15.37:32902) 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f63767fc640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 0x7f636c0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6384a04640 1 -- 172.21.15.37:0/4060303536 learned_addr learned my addr 172.21.15.37:0/4060303536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6385205640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a16ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6385a06640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 0x7f636c0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.846+0000 7f6384a04640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 -- 172.21.15.37:0/4060303536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 msgr2=0x558646a16ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a16ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 -- 172.21.15.37:0/4060303536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 msgr2=0x558646a194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 -- 172.21.15.37:0/4060303536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586469390d0 con 0x558646a0bf00 2024-04-26T18:25:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f63767fc640 1 -- 172.21.15.37:0/1282891368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f63780f2110 con 0x7f638010f2f0 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6385205640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a16ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6384a04640 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f637000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1907fa6d5174c77b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f6385a06640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 0x7f636c0a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63780ef780 tx=0x7f637802a8c0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f636b7fe640 1 -- 172.21.15.37:0/4060303536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370019030 con 0x558646a0bf00 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f636b7fe640 1 -- 172.21.15.37:0/4060303536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63700035f0 con 0x558646a0bf00 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.847+0000 7f636b7fe640 1 -- 172.21.15.37:0/4060303536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370011720 con 0x558646a0bf00 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- 172.21.15.37:0/4060303536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 msgr2=0x558646a0c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f63700058c0 tx=0x7f637000b6b0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- 172.21.15.37:0/4060303536 shutdown_connections 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a16ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 --2- 172.21.15.37:0/4060303536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a0c2e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- 172.21.15.37:0/4060303536 >> 172.21.15.37:0/4060303536 conn(0x558646a08410 msgr2=0x558646a087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- 172.21.15.37:0/4060303536 shutdown_connections 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- 172.21.15.37:0/4060303536 wait complete. 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 -- start start 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.848+0000 7f638a178740 1 Processor -- start 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a14ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558646a19d10 con 0x558646a0bf00 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586468f9b60 con 0x558646a0c820 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558646a19e90 con 0x558646a170e0 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f638a178740 10 client.0 _ll_register_callbacks cb 0x55864691e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32906/0 (socket says 172.21.15.37:32906) 2024-04-26T18:25:14.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 -- 172.21.15.37:0/943233959 learned_addr learned my addr 172.21.15.37:0/943233959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6385205640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 -- 172.21.15.37:0/943233959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 msgr2=0x558646a14ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a14ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 -- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 msgr2=0x558646a14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 -- 172.21.15.37:0/943233959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558646684210 con 0x558646a0bf00 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6385205640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6384a04640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6370005890 tx=0x7f63700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858c67dbf3bdee20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.849+0000 7f6368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370019070 con 0x558646a0bf00 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63700035f0 con 0x558646a0bf00 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f63700042c0 con 0x558646a0bf00 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63700044c0 con 0x558646a0bf00 2024-04-26T18:25:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 10 client.15420 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558646750620 con 0x558646a0bf00 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63700035f0 con 0x558646a0bf00 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6370026070 con 0x558646a0bf00 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 20 client.15420 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a270d0 0x558646a294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 4 client.15420 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.850+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558646980070 con 0x558646a270d0 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.851+0000 7f6385205640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a270d0 0x558646a294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.851+0000 7f6368ff9640 10 client.15420 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.851+0000 7f6385205640 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a270d0 0x558646a294c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee28c62fb55fe32 server_cookie=de6e0fce9deaf37c in_seq=0 out_seq=0 2024-04-26T18:25:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f6368ff9640 1 -- 172.21.15.37:0/943233959 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558646980070 con 0x558646a270d0 2024-04-26T18:25:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f6368ff9640 10 client.15420 handle_command_reply: tid=0 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 client.15420 shutdown 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 msgr2=0x558646a14040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6370005890 tx=0x7f63700039d0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 shutdown_connections 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a270d0 0x558646a294c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a14ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f6368ff9640 0 client.15420 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 --2- 172.21.15.37:0/943233959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a14040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 >> 172.21.15.37:0/943233959 conn(0x558646a08410 msgr2=0x558646a0e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 shutdown_connections 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 1 -- 172.21.15.37:0/943233959 wait complete. 2024-04-26T18:25:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.852+0000 7f638a178740 20 client.15420 trim_cache size 0 max 16384 2024-04-26T18:25:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 Processor -- start 2024-04-26T18:25:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 -- start start 2024-04-26T18:25:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a22bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a2ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63801b3040 con 0x558646a0bf00 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63801b0910 con 0x558646a170e0 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.867+0000 7f638a178740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ae760 con 0x558646a0c820 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32912/0 (socket says 172.21.15.37:32912) 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 -- 172.21.15.37:0/2113727608 learned_addr learned my addr 172.21.15.37:0/2113727608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6385205640 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a22bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 -- 172.21.15.37:0/2113727608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 msgr2=0x558646a22bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a22bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 -- 172.21.15.37:0/2113727608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 msgr2=0x558646a2ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a2ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 -- 172.21.15.37:0/2113727608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558646a0d1c0 con 0x558646a0bf00 2024-04-26T18:25:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f6384a04640 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f637001ebb0 tx=0x7f6370005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8cb8a5d92cacac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f636affd640 1 -- 172.21.15.37:0/2113727608 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370019040 con 0x558646a0bf00 2024-04-26T18:25:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f636affd640 1 -- 172.21.15.37:0/2113727608 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637001a040 con 0x558646a0bf00 2024-04-26T18:25:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.868+0000 7f636affd640 1 -- 172.21.15.37:0/2113727608 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63700040c0 con 0x558646a0bf00 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- 172.21.15.37:0/2113727608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 msgr2=0x558646a07d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f637001ebb0 tx=0x7f6370005760 comp rx=0 tx=0).stop 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- 172.21.15.37:0/2113727608 shutdown_connections 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a170e0 0x558646a2ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a0c820 0x558646a22bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 --2- 172.21.15.37:0/2113727608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a07d30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- 172.21.15.37:0/2113727608 >> 172.21.15.37:0/2113727608 conn(0x558646a08410 msgr2=0x558646a16100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- 172.21.15.37:0/2113727608 shutdown_connections 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- 172.21.15.37:0/2113727608 wait complete. 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 -- start start 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.869+0000 7f638a178740 1 Processor -- start 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a1f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a1c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63801aedd0 con 0x558646a0bf00 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63801b0240 con 0x558646a0c820 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63801af4a0 con 0x558646a170e0 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 10 client.0 _ll_register_callbacks cb 0x7f6370013420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32926/0 (socket says 172.21.15.37:32926) 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 -- 172.21.15.37:0/2489769205 learned_addr learned my addr 172.21.15.37:0/2489769205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a1c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6385205640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a1f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 -- 172.21.15.37:0/2489769205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 msgr2=0x558646a1c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a1c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 -- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 msgr2=0x558646a1f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a1f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 -- 172.21.15.37:0/2489769205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558646a29920 con 0x558646a0bf00 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6384a04640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f637001ea30 tx=0x7f63700039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b7dceaf1348419 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6368ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370019050 con 0x558646a0bf00 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f637000bbc0 con 0x558646a0bf00 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.870+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637001a040 con 0x558646a0bf00 2024-04-26T18:25:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6370022040 con 0x558646a0bf00 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 10 client.15426 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558646a8dc60 con 0x558646a0bf00 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637001a040 con 0x558646a0bf00 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6370026070 con 0x558646a0bf00 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 20 client.15426 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a8e2c0 0x558646a906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 4 client.15426 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558646979ee0 con 0x558646a8e2c0 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.871+0000 7f6385205640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a8e2c0 0x558646a906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.872+0000 7f6368ff9640 10 client.15426 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.872+0000 7f6385205640 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a8e2c0 0x558646a906e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62414a59d01705c server_cookie=f68b9444313bb855 in_seq=0 out_seq=0 2024-04-26T18:25:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f6368ff9640 1 -- 172.21.15.37:0/2489769205 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558646979ee0 con 0x558646a8e2c0 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f6368ff9640 10 client.15426 handle_command_reply: tid=0 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 client.15426 shutdown 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 msgr2=0x558646a1f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f637001ea30 tx=0x7f63700039d0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 shutdown_connections 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558646a8e2c0 0x558646a906e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558646a0c820 0x558646a1f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558646a170e0 0x558646a1c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 --2- 172.21.15.37:0/2489769205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558646a0bf00 0x558646a1f0e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f6368ff9640 0 client.15426 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 >> 172.21.15.37:0/2489769205 conn(0x558646a08410 msgr2=0x558646a0d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 shutdown_connections 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 1 -- 172.21.15.37:0/2489769205 wait complete. 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.879+0000 7f638a178740 20 client.15426 trim_cache size 0 max 16384 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:25:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 msgr2=0x7f636c0a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 0x7f636c0a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63780ef780 tx=0x7f637802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 msgr2=0x7f638019fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f63780319f0 tx=0x7f6378031a20 comp rx=0 tx=0).stop 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 shutdown_connections 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f636c0a3f70 0x7f636c0a6430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f638010f2f0 0x7f638019fb20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6380102060 0x7f638019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 --2- 172.21.15.37:0/1282891368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6380101740 0x7f638019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 >> 172.21.15.37:0/1282891368 conn(0x7f63800789b0 msgr2=0x7f6380110780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 shutdown_connections 2024-04-26T18:25:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:14.880+0000 7f6387469640 1 -- 172.21.15.37:0/1282891368 wait complete. 2024-04-26T18:25:15.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:15 smithi037 ceph-mon[29171]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:25:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:15 smithi120 ceph-mon[32244]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:25:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:15 smithi170 ceph-mon[32697]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:25:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:17 smithi120 ceph-mon[32244]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-26T18:25:17.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:17 smithi037 ceph-mon[29171]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-26T18:25:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:17 smithi170 ceph-mon[32697]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-26T18:25:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:19 smithi120 ceph-mon[32244]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:25:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:19 smithi170 ceph-mon[32697]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:25:19.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:19 smithi037 ceph-mon[29171]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:25:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:21 smithi120 ceph-mon[32244]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:21 smithi170 ceph-mon[32697]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:21.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:21 smithi037 ceph-mon[29171]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:23 smithi120 ceph-mon[32244]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-26T18:25:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:23 smithi170 ceph-mon[32697]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-26T18:25:23.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:23 smithi037 ceph-mon[29171]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-26T18:25:24.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:24 smithi120 ceph-mon[32244]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:24.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:24 smithi170 ceph-mon[32697]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:24.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:24 smithi037 ceph-mon[29171]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-26T18:25:26.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:26 smithi037 ceph-mon[29171]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:25:26.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:26.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:26.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:26 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:26 smithi120 ceph-mon[32244]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:25:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:26 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:26 smithi170 ceph-mon[32697]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:25:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:26 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.974+0000 7f8abbcfc640 1 -- 172.21.15.37:0/344988190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 msgr2=0x7f8ab410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.974+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/344988190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab410e6e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009cd0 tx=0x7f8aac02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 -- 172.21.15.37:0/344988190 shutdown_connections 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/344988190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4101860 0x7f8ab4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/344988190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab410e6e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/344988190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4107820 0x7f8ab4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 -- 172.21.15.37:0/344988190 >> 172.21.15.37:0/344988190 conn(0x7f8ab40789b0 msgr2=0x7f8ab4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 -- 172.21.15.37:0/344988190 shutdown_connections 2024-04-26T18:25:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 -- 172.21.15.37:0/344988190 wait complete. 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.975+0000 7f8abbcfc640 1 Processor -- start 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 -- start start 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4101860 0x7f8ab4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4107820 0x7f8ab41a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111ae0 con 0x7f8ab4102220 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111960 con 0x7f8ab4107820 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8abbcfc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111c60 con 0x7f8ab4101860 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49284/0 (socket says 172.21.15.37:49284) 2024-04-26T18:25:27.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 -- 172.21.15.37:0/1364859338 learned_addr learned my addr 172.21.15.37:0/1364859338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4101860 0x7f8ab4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8aba299640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4107820 0x7f8ab41a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 -- 172.21.15.37:0/1364859338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4101860 msgr2=0x7f8ab4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4101860 0x7f8ab4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 -- 172.21.15.37:0/1364859338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4107820 msgr2=0x7f8ab41a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4107820 0x7f8ab41a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.976+0000 7f8ab9297640 1 -- 172.21.15.37:0/1364859338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab41a5380 con 0x7f8ab4102220 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8ab9297640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d950 tx=0x7f8aa400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8aaaffd640 1 -- 172.21.15.37:0/1364859338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4017070 con 0x7f8ab4102220 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8aaaffd640 1 -- 172.21.15.37:0/1364859338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa400de80 con 0x7f8ab4102220 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab41a5550 con 0x7f8ab4102220 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8aaaffd640 1 -- 172.21.15.37:0/1364859338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400f620 con 0x7f8ab4102220 2024-04-26T18:25:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.977+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a5ac0 con 0x7f8ab4102220 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.979+0000 7f8aaaffd640 1 -- 172.21.15.37:0/1364859338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8aa4009c30 con 0x7f8ab4102220 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.979+0000 7f8abea14740 1 Processor -- start 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 -- start start 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 0x5608f0a40ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608f099c460 con 0x5608f0a410e0 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608f0956590 con 0x5608f0a36820 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8abea14740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608f0998420 con 0x5608f0a35f00 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49294/0 (socket says 172.21.15.37:49294) 2024-04-26T18:25:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8ab9297640 1 -- 172.21.15.37:0/4240030249 learned_addr learned my addr 172.21.15.37:0/4240030249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 0x5608f0a40ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8aaaffd640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 0x7f8aa00a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.980+0000 7f8aba299640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8aba299640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 0x7f8aa00a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 -- 172.21.15.37:0/4240030249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 msgr2=0x5608f0a362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8aaaffd640 1 -- 172.21.15.37:0/1364859338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8aa40d05e0 con 0x7f8ab4102220 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 -- 172.21.15.37:0/4240030249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 msgr2=0x5608f0a434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 -- 172.21.15.37:0/4240030249 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f09630d0 con 0x5608f0a36820 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9297640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 0x5608f0a40ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000ab70 tx=0x7f8ab000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b263079dbd88d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8aba299640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 0x7f8aa00a64b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fe0 tx=0x7f8aac005da0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8a9ffff640 1 -- 172.21.15.37:0/4240030249 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0005220 con 0x5608f0a36820 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8a9ffff640 1 -- 172.21.15.37:0/4240030249 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab0004c50 con 0x5608f0a36820 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.981+0000 7f8a9ffff640 1 -- 172.21.15.37:0/4240030249 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0018730 con 0x5608f0a36820 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 -- 172.21.15.37:0/4240030249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 msgr2=0x5608f0a40ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 0x5608f0a40ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000ab70 tx=0x7f8ab000f790 comp rx=0 tx=0).stop 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 -- 172.21.15.37:0/4240030249 shutdown_connections 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a36820 0x5608f0a40ba0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 --2- 172.21.15.37:0/4240030249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 -- 172.21.15.37:0/4240030249 >> 172.21.15.37:0/4240030249 conn(0x5608f0a32410 msgr2=0x5608f0a327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 -- 172.21.15.37:0/4240030249 shutdown_connections 2024-04-26T18:25:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.982+0000 7f8abea14740 1 -- 172.21.15.37:0/4240030249 wait complete. 2024-04-26T18:25:27.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:27 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:27.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:27 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:27.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:27 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 -- start start 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 Processor -- start 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5608f0a43d10 con 0x5608f0a35f00 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5608f0923b60 con 0x5608f0a410e0 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5608f0a43e90 con 0x5608f0a36820 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8abea14740 10 client.0 _ll_register_callbacks cb 0x5608f09482b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8ab9a98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8ab9a98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49300/0 (socket says 172.21.15.37:49300) 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.983+0000 7f8ab9a98640 1 -- 172.21.15.37:0/2315528823 learned_addr learned my addr 172.21.15.37:0/2315528823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8aba299640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9297640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 -- 172.21.15.37:0/2315528823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 msgr2=0x5608f0a3e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a3e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 -- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 msgr2=0x5608f0a3ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 -- 172.21.15.37:0/2315528823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f06ae210 con 0x5608f0a35f00 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9297640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a3e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005690 tx=0x7f8ab00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a00eeba625d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8a9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0027070 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8ab00052a0 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab0004c50 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ab0033050 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8abea14740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.984+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608f077a620 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab0004c50 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8ab0023430 con 0x5608f0a35f00 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 20 client.15438 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0a53480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 4 client.15438 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f09aa070 con 0x5608f0a51090 2024-04-26T18:25:27.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.985+0000 7f8ab9297640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0a53480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8ab9297640 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0a53480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4640e2c5830ab2f server_cookie=ff22e26699f3fe9e in_seq=0 out_seq=0 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8a9d7fa640 10 client.15438 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/2315528823 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608f09aa070 con 0x5608f0a51090 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8a9d7fa640 10 client.15438 handle_command_reply: tid=0 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 client.15438 shutdown 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 msgr2=0x5608f0a3df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005690 tx=0x7f8ab00050b0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 shutdown_connections 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0a53480 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8a9d7fa640 0 client.15438 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a3e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.986+0000 7f8abea14740 1 --2- 172.21.15.37:0/2315528823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a35f00 0x5608f0a3df80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.987+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 >> 172.21.15.37:0/2315528823 conn(0x5608f0a32410 msgr2=0x5608f0a38140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.987+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 shutdown_connections 2024-04-26T18:25:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.987+0000 7f8abea14740 1 -- 172.21.15.37:0/2315528823 wait complete. 2024-04-26T18:25:27.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:27.987+0000 7f8abea14740 20 client.15438 trim_cache size 0 max 16384 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 Processor -- start 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 -- start start 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 0x5608f0a3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b0d20 con 0x5608f0a410e0 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41ae5f0 con 0x5608f0a35f00 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8abea14740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41ac440 con 0x5608f0a36820 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49310/0 (socket says 172.21.15.37:49310) 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 -- 172.21.15.37:0/92647864 learned_addr learned my addr 172.21.15.37:0/92647864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 0x5608f0a3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8ab9297640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 -- 172.21.15.37:0/92647864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 msgr2=0x5608f0a58b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a58b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 -- 172.21.15.37:0/92647864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 msgr2=0x5608f0a3d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 0x5608f0a3d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8aba299640 1 -- 172.21.15.37:0/92647864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f0a373d0 con 0x5608f0a410e0 2024-04-26T18:25:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.002+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 0x5608f0a3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.003+0000 7f8aba299640 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009cd0 tx=0x7f8aac009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285d3f0b626ad0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.003+0000 7f8a9f7fe640 1 -- 172.21.15.37:0/92647864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac031c30 con 0x5608f0a410e0 2024-04-26T18:25:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.003+0000 7f8a9f7fe640 1 -- 172.21.15.37:0/92647864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aac036040 con 0x5608f0a410e0 2024-04-26T18:25:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.003+0000 7f8a9f7fe640 1 -- 172.21.15.37:0/92647864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0309c0 con 0x5608f0a410e0 2024-04-26T18:25:28.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:27 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:28.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:27 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:28.006 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:27 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- 172.21.15.37:0/92647864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 msgr2=0x5608f0a59080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009cd0 tx=0x7f8aac009380 comp rx=0 tx=0).stop 2024-04-26T18:25:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- 172.21.15.37:0/92647864 shutdown_connections 2024-04-26T18:25:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a410e0 0x5608f0a59080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a36820 0x5608f0a58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 --2- 172.21.15.37:0/92647864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a35f00 0x5608f0a3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- 172.21.15.37:0/92647864 >> 172.21.15.37:0/92647864 conn(0x5608f0a32410 msgr2=0x5608f0a327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- 172.21.15.37:0/92647864 shutdown_connections 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- 172.21.15.37:0/92647864 wait complete. 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 -- start start 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.004+0000 7f8abea14740 1 Processor -- start 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a3ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41acab0 con 0x5608f0a36820 2024-04-26T18:25:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41adf20 con 0x5608f0a410e0 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41ad180 con 0x5608f0a35f00 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8abea14740 10 client.0 _ll_register_callbacks cb 0x7f8aa40985d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49322/0 (socket says 172.21.15.37:49322) 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 -- 172.21.15.37:0/770549228 learned_addr learned my addr 172.21.15.37:0/770549228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8aba299640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a3ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 -- 172.21.15.37:0/770549228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 msgr2=0x5608f0a3ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a3ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 -- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 msgr2=0x5608f0a3b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.005+0000 7f8ab9297640 1 -- 172.21.15.37:0/770549228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f0a38d70 con 0x5608f0a36820 2024-04-26T18:25:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8ab9297640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8aa409c2c0 tx=0x7f8aa40d2a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f0a48f51c774b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4020040 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8aa40d2b50 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aac036040 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8aa409ca30 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8abea14740 10 client.15444 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608f0ab7480 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aac036040 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.006+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8aa4098080 con 0x5608f0a36820 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 20 client.15444 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0ab9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 4 client.15444 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608f09a3ee0 con 0x5608f0a51090 2024-04-26T18:25:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0ab9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8ab9a98640 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0ab9c70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9772138927355db server_cookie=10a9a3239f86b2c in_seq=0 out_seq=0 2024-04-26T18:25:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.007+0000 7f8a9d7fa640 10 client.15444 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8a9d7fa640 1 -- 172.21.15.37:0/770549228 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608f09a3ee0 con 0x5608f0a51090 2024-04-26T18:25:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8a9d7fa640 10 client.15444 handle_command_reply: tid=0 2024-04-26T18:25:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 client.15444 shutdown 2024-04-26T18:25:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 msgr2=0x5608f0a3b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8aa409c2c0 tx=0x7f8aa40d2a70 comp rx=0 tx=0).stop 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 shutdown_connections 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5608f0a51090 0x5608f0ab9c70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5608f0a410e0 0x5608f0a3b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8a9d7fa640 0 client.15444 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5608f0a36820 0x5608f0a3b0b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 --2- 172.21.15.37:0/770549228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5608f0a35f00 0x5608f0a3ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 >> 172.21.15.37:0/770549228 conn(0x5608f0a32410 msgr2=0x5608f0a38140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 shutdown_connections 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 1 -- 172.21.15.37:0/770549228 wait complete. 2024-04-26T18:25:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.011+0000 7f8abea14740 20 client.15444 trim_cache size 0 max 16384 2024-04-26T18:25:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:25:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:25:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:25:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 msgr2=0x7f8aa00a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 0x7f8aa00a64b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fe0 tx=0x7f8aac005da0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 msgr2=0x7f8ab419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d950 tx=0x7f8aa400dd50 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 shutdown_connections 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8aa00a3ff0 0x7f8aa00a64b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ab4107820 0x7f8ab41a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ab4102220 0x7f8ab419a300 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 --2- 172.21.15.37:0/1364859338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ab4101860 0x7f8ab4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 >> 172.21.15.37:0/1364859338 conn(0x7f8ab40789b0 msgr2=0x7f8ab4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 shutdown_connections 2024-04-26T18:25:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:28.012+0000 7f8abbcfc640 1 -- 172.21.15.37:0/1364859338 wait complete. 2024-04-26T18:25:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:27 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:28.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:27 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:28.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:27 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:25:29.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:29 smithi170 ceph-mon[32697]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:29.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:29 smithi037 ceph-mon[29171]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:29 smithi120 ceph-mon[32244]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:31.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:31 smithi170 ceph-mon[32697]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:31 smithi037 ceph-mon[29171]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:31 smithi120 ceph-mon[32244]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:25:33.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:33 smithi170 ceph-mon[32697]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:33.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:33 smithi037 ceph-mon[29171]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:33 smithi120 ceph-mon[32244]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:35.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:35 smithi170 ceph-mon[32697]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:25:35.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:35 smithi037 ceph-mon[29171]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:25:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:35 smithi120 ceph-mon[32244]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:25:37.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:37 smithi170 ceph-mon[32697]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:37.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:37 smithi037 ceph-mon[29171]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:37.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:37 smithi120 ceph-mon[32244]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:25:39.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:39 smithi170 ceph-mon[32697]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:39.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:39 smithi037 ceph-mon[29171]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:39 smithi120 ceph-mon[32244]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 -- 172.21.15.37:0/2414710450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834108950 msgr2=0x7fc834108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 --2- 172.21.15.37:0/2414710450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834108950 0x7fc834108d30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009cd0 tx=0x7fc82c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 -- 172.21.15.37:0/2414710450 shutdown_connections 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 --2- 172.21.15.37:0/2414710450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834103300 0x7fc83410f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 --2- 172.21.15.37:0/2414710450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834102940 0x7fc834102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 --2- 172.21.15.37:0/2414710450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834108950 0x7fc834108d30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 -- 172.21.15.37:0/2414710450 >> 172.21.15.37:0/2414710450 conn(0x7fc834079dc0 msgr2=0x7fc83407a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 -- 172.21.15.37:0/2414710450 shutdown_connections 2024-04-26T18:25:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.146+0000 7fc83916a640 1 -- 172.21.15.37:0/2414710450 wait complete. 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.147+0000 7fc83916a640 1 Processor -- start 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.147+0000 7fc83916a640 1 -- start start 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834102940 0x7fc834074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834108950 0x7fc8340756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc834112c60 con 0x7fc834103300 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc834112ae0 con 0x7fc834102940 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83916a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc834112de0 con 0x7fc834108950 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50700/0 (socket says 172.21.15.37:50700) 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 -- 172.21.15.37:0/1581313257 learned_addr learned my addr 172.21.15.37:0/1581313257 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834102940 0x7fc834074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83359e640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834108950 0x7fc8340756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 -- 172.21.15.37:0/1581313257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834108950 msgr2=0x7fc8340756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834108950 0x7fc8340756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 -- 172.21.15.37:0/1581313257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834102940 msgr2=0x7fc834074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834102940 0x7fc834074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.148+0000 7fc83259c640 1 -- 172.21.15.37:0/1581313257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8341a75a0 con 0x7fc834103300 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc83259c640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d770 tx=0x7fc81c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc82bfff640 1 -- 172.21.15.37:0/1581313257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c017070 con 0x7fc834103300 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341a7890 con 0x7fc834103300 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc82bfff640 1 -- 172.21.15.37:0/1581313257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c0054d0 con 0x7fc834103300 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341a7de0 con 0x7fc834103300 2024-04-26T18:25:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.149+0000 7fc82bfff640 1 -- 172.21.15.37:0/1581313257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c004c30 con 0x7fc834103300 2024-04-26T18:25:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.151+0000 7fc83be84740 1 Processor -- start 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 -- start start 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b655820 0x55571b65fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b6600e0 0x55571b6624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55571b5bb460 con 0x55571b654f00 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55571b575590 con 0x55571b6600e0 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83be84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55571b5b7420 con 0x55571b655820 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50708/0 (socket says 172.21.15.37:50708) 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc83359e640 1 -- 172.21.15.37:0/1643342086 learned_addr learned my addr 172.21.15.37:0/1643342086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b655820 0x55571b65fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.152+0000 7fc82bfff640 1 -- 172.21.15.37:0/1581313257 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc81c004eb0 con 0x7fc834103300 2024-04-26T18:25:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 -- 172.21.15.37:0/1643342086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b655820 msgr2=0x55571b65fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b655820 0x55571b65fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 -- 172.21.15.37:0/1643342086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b6600e0 msgr2=0x55571b6624d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b6600e0 0x55571b6624d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 -- 172.21.15.37:0/1643342086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571b5820d0 con 0x55571b654f00 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc83359e640 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc824009c10 tx=0x7fc824011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06e26d51db99edb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc8297fa640 1 -- 172.21.15.37:0/1643342086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82401e070 con 0x55571b654f00 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc8297fa640 1 -- 172.21.15.37:0/1643342086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc824019ae0 con 0x55571b654f00 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.153+0000 7fc82bfff640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 0x7fc8200a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc82bfff640 1 -- 172.21.15.37:0/1581313257 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fc81c0d04e0 con 0x7fc834103300 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 0x7fc8200a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc8297fa640 1 -- 172.21.15.37:0/1643342086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400e8f0 con 0x55571b654f00 2024-04-26T18:25:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 -- 172.21.15.37:0/1643342086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 msgr2=0x55571b6552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc824009c10 tx=0x7fc824011810 comp rx=0 tx=0).stop 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 0x7fc8200a64b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009ca0 tx=0x7fc82c004a80 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 -- 172.21.15.37:0/1643342086 shutdown_connections 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b6600e0 0x55571b6624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b655820 0x55571b65fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 --2- 172.21.15.37:0/1643342086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b6552e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.154+0000 7fc83be84740 1 -- 172.21.15.37:0/1643342086 >> 172.21.15.37:0/1643342086 conn(0x55571b651410 msgr2=0x55571b6517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- 172.21.15.37:0/1643342086 shutdown_connections 2024-04-26T18:25:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- 172.21.15.37:0/1643342086 wait complete. 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- start start 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 Processor -- start 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 0x55571b65d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55571b662d60 con 0x55571b654f00 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55571b542b60 con 0x55571b655820 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.155+0000 7fc83be84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55571b662ee0 con 0x55571b6600e0 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc83be84740 10 client.0 _ll_register_callbacks cb 0x55571b5672b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50712/0 (socket says 172.21.15.37:50712) 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 -- 172.21.15.37:0/3892603557 learned_addr learned my addr 172.21.15.37:0/3892603557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc83359e640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc83259c640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 0x55571b65d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 -- 172.21.15.37:0/3892603557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 msgr2=0x55571b65dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 -- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 msgr2=0x55571b65d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 0x55571b65d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 -- 172.21.15.37:0/3892603557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571b2cd210 con 0x55571b654f00 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc83259c640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 0x55571b65d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009cd0 tx=0x7fc82c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e76cd5ff994fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c056060 con 0x55571b654f00 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fc82c041030 con 0x55571b654f00 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.156+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc824019ae0 con 0x55571b654f00 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc82c003fd0 con 0x55571b654f00 2024-04-26T18:25:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55571b399620 con 0x55571b654f00 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc824019ae0 con 0x55571b654f00 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc82c030070 con 0x55571b654f00 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 20 client.15456 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6702b0 0x55571b6726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 4 client.15456 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.157+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55571b5c9070 con 0x55571b6702b0 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.158+0000 7fc83259c640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6702b0 0x55571b6726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.158+0000 7fc81a7fc640 10 client.15456 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.158+0000 7fc83259c640 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6702b0 0x55571b6726a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d011ed51d1a2f59f server_cookie=ec7994fd92668e7a in_seq=0 out_seq=0 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3892603557 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55571b5c9070 con 0x55571b6702b0 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc81a7fc640 10 client.15456 handle_command_reply: tid=0 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 client.15456 shutdown 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 msgr2=0x55571b65d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009cd0 tx=0x7fc82c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 shutdown_connections 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6702b0 0x55571b6726a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b655820 0x55571b65d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc81a7fc640 0 client.15456 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 --2- 172.21.15.37:0/3892603557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b654f00 0x55571b65d040 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 >> 172.21.15.37:0/3892603557 conn(0x55571b651410 msgr2=0x55571b657140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 shutdown_connections 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 1 -- 172.21.15.37:0/3892603557 wait complete. 2024-04-26T18:25:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.159+0000 7fc83be84740 20 client.15456 trim_cache size 0 max 16384 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.188+0000 7fc83be84740 1 Processor -- start 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.188+0000 7fc83be84740 1 -- start start 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b66bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341b2fb0 con 0x55571b655820 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341b0880 con 0x55571b654f00 2024-04-26T18:25:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83be84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341ae6d0 con 0x55571b6600e0 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50726/0 (socket says 172.21.15.37:50726) 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 -- 172.21.15.37:0/1760238253 learned_addr learned my addr 172.21.15.37:0/1760238253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b66bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83359e640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 -- 172.21.15.37:0/1760238253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 msgr2=0x55571b65ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 -- 172.21.15.37:0/1760238253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 msgr2=0x55571b66bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b66bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc83259c640 1 -- 172.21.15.37:0/1760238253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571b657e00 con 0x55571b655820 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.189+0000 7fc832d9d640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b66bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc83259c640 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009610 tx=0x7fc81c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b692579fac7d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc828ff9640 1 -- 172.21.15.37:0/1760238253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c020040 con 0x55571b655820 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc828ff9640 1 -- 172.21.15.37:0/1760238253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c096e50 con 0x55571b655820 2024-04-26T18:25:41.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc828ff9640 1 -- 172.21.15.37:0/1760238253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c098950 con 0x55571b655820 2024-04-26T18:25:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc83be84740 1 -- 172.21.15.37:0/1760238253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 msgr2=0x55571b6578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.190+0000 7fc83be84740 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009610 tx=0x7fc81c009e90 comp rx=0 tx=0).stop 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- 172.21.15.37:0/1760238253 shutdown_connections 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b65ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6578c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- 172.21.15.37:0/1760238253 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b66bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- 172.21.15.37:0/1760238253 >> 172.21.15.37:0/1760238253 conn(0x55571b651410 msgr2=0x55571b65e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- 172.21.15.37:0/1760238253 shutdown_connections 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- 172.21.15.37:0/1760238253 wait complete. 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- start start 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 Processor -- start 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b668480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b663d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341aed40 con 0x55571b655820 2024-04-26T18:25:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341b01b0 con 0x55571b654f00 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341af410 con 0x55571b6600e0 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.191+0000 7fc83be84740 10 client.0 _ll_register_callbacks cb 0x7fc82c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b663d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50738/0 (socket says 172.21.15.37:50738) 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 -- 172.21.15.37:0/3837288650 learned_addr learned my addr 172.21.15.37:0/3837288650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b668480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 -- 172.21.15.37:0/3837288650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 msgr2=0x55571b663d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b663d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 -- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 msgr2=0x55571b668480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b668480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 -- 172.21.15.37:0/3837288650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571b664560 con 0x55571b655820 2024-04-26T18:25:41.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83359e640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b663d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83259c640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc81c098740 tx=0x7fc81c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cb63b488d08bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0b9020 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fc81c098950 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81c096e50 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc81c020070 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.192+0000 7fc83be84740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55571b6d6750 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc81c096e50 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc81c0cdcf0 con 0x55571b655820 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 20 client.15462 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6d6e40 0x55571b6d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 4 client.15462 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55571b5c2ee0 con 0x55571b6d6e40 2024-04-26T18:25:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.193+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6d6e40 0x55571b6d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.194+0000 7fc81a7fc640 10 client.15462 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.194+0000 7fc832d9d640 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6d6e40 0x55571b6d9260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2bb87c680b286fe server_cookie=3b00d8747d02eeff in_seq=0 out_seq=0 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.197+0000 7fc81a7fc640 1 -- 172.21.15.37:0/3837288650 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55571b5c2ee0 con 0x55571b6d6e40 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.197+0000 7fc81a7fc640 10 client.15462 handle_command_reply: tid=0 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.197+0000 7fc83be84740 1 client.15462 shutdown 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 msgr2=0x55571b6637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc81c098740 tx=0x7fc81c009f00 comp rx=0 tx=0).stop 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 shutdown_connections 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55571b6d6e40 0x55571b6d9260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55571b6600e0 0x55571b663d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc81a7fc640 0 client.15462 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55571b655820 0x55571b6637f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 --2- 172.21.15.37:0/3837288650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55571b654f00 0x55571b668480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 >> 172.21.15.37:0/3837288650 conn(0x55571b651410 msgr2=0x55571b65f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 shutdown_connections 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 1 -- 172.21.15.37:0/3837288650 wait complete. 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83be84740 20 client.15462 trim_cache size 0 max 16384 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:25:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 msgr2=0x7fc8200a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.198+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 0x7fc8200a64b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009ca0 tx=0x7fc82c004a80 comp rx=0 tx=0).stop 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 msgr2=0x7fc834075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d770 tx=0x7fc81c00dc30 comp rx=0 tx=0).stop 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 shutdown_connections 2024-04-26T18:25:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fc8200a3ff0 0x7fc8200a64b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc834108950 0x7fc8340756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc834103300 0x7fc834075170 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 --2- 172.21.15.37:0/1581313257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc834102940 0x7fc834074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 >> 172.21.15.37:0/1581313257 conn(0x7fc834079dc0 msgr2=0x7fc834100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 shutdown_connections 2024-04-26T18:25:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:41.199+0000 7fc83916a640 1 -- 172.21.15.37:0/1581313257 wait complete. 2024-04-26T18:25:41.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:41 smithi170 ceph-mon[32697]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:41 smithi037 ceph-mon[29171]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:41 smithi120 ceph-mon[32244]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:43.305 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:43 smithi120 ceph-mon[32244]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:43.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:43 smithi170 ceph-mon[32697]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:43.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:25:43.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:43 smithi037 ceph-mon[29171]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:43.709 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:45.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:45 smithi037 ceph-mon[29171]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:25:45.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:45 smithi170 ceph-mon[32697]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:25:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:45 smithi120 ceph-mon[32244]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:25:45.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 -- 172.21.15.37:0/1543373129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7238102350 msgr2=0x7f72381027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 --2- 172.21.15.37:0/1543373129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7238102350 0x7f72381027d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f722c009990 tx=0x7f722c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 -- 172.21.15.37:0/1543373129 shutdown_connections 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 --2- 172.21.15.37:0/1543373129 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102d10 0x7f723810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 --2- 172.21.15.37:0/1543373129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72381083c0 0x7f72381087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 --2- 172.21.15.37:0/1543373129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7238102350 0x7f72381027d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 -- 172.21.15.37:0/1543373129 >> 172.21.15.37:0/1543373129 conn(0x7f7238079360 msgr2=0x7f7238079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:45.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 -- 172.21.15.37:0/1543373129 shutdown_connections 2024-04-26T18:25:45.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.561+0000 7f723e746700 1 -- 172.21.15.37:0/1543373129 wait complete. 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.562+0000 7f723e746700 1 Processor -- start 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.562+0000 7f723e746700 1 -- start start 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7238102d10 0x7f723819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 0x7f72381999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7238113630 con 0x7f72381083c0 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72381134b0 con 0x7f7238102350 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723e746700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72381137b0 con 0x7f7238102d10 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:59514/0 (socket says 172.21.15.37:59514) 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 -- 172.21.15.37:0/1775382873 learned_addr learned my addr 172.21.15.37:0/1775382873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f72377fe700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7238102d10 0x7f723819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723cce3700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 0x7f72381999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:45.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 -- 172.21.15.37:0/1775382873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7238102d10 msgr2=0x7f723819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7238102d10 0x7f723819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 -- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 msgr2=0x7f72381999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 0x7f72381999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f7237fff700 1 -- 172.21.15.37:0/1775382873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723819a110 con 0x7f7238102350 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.563+0000 7f723cce3700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 0x7f72381999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:45.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f7237fff700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f722800d980 tx=0x7f722800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228017070 con 0x7f7238102350 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723819a400 con 0x7f7238102350 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7228005020 con 0x7f7238102350 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72381a66c0 con 0x7f7238102350 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.564+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228009b50 con 0x7f7238102350 2024-04-26T18:25:45.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.565+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7224005330 con 0x7f7238102350 2024-04-26T18:25:45.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.567+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f72280051d0 con 0x7f7238102350 2024-04-26T18:25:45.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.568+0000 7f72357fa700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 0x7f72200a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:45.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.568+0000 7f72377fe700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 0x7f72200a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:45.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.568+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f72280d0840 con 0x7f7238102350 2024-04-26T18:25:45.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.569+0000 7f72377fe700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 0x7f72200a64a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f722c0098e0 tx=0x7f722c005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:45.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.573+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7228020020 con 0x7f7238102350 2024-04-26T18:25:45.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.841+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72240058f0 con 0x7f7238102350 2024-04-26T18:25:45.842 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:25:45.842 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:25:45.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.842+0000 7f72357fa700 1 -- 172.21.15.37:0/1775382873 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f722809cc90 con 0x7f7238102350 2024-04-26T18:25:45.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 msgr2=0x7f72200a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:45.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 0x7f72200a64a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f722c0098e0 tx=0x7f722c005960 comp rx=0 tx=0).stop 2024-04-26T18:25:45.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 msgr2=0x7f723819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:45.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f722800d980 tx=0x7f722800dc90 comp rx=0 tx=0).stop 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 shutdown_connections 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72381083c0 0x7f72381999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f72200a3fe0 0x7f72200a64a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7238102350 0x7f723819f7f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 --2- 172.21.15.37:0/1775382873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7238102d10 0x7f723819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.844+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 >> 172.21.15.37:0/1775382873 conn(0x7f7238079360 msgr2=0x7f72381011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.845+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 shutdown_connections 2024-04-26T18:25:45.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:45.845+0000 7f723e746700 1 -- 172.21.15.37:0/1775382873 wait complete. 2024-04-26T18:25:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:25:46.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:46 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1775382873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:46.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:46 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1775382873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:46.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:46 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1775382873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:46.736 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id b2739c93 --release --await' 2024-04-26T18:25:46.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db --set-id b2739c93 --release --await 2024-04-26T18:25:47.144 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:47.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:47 smithi170 ceph-mon[32697]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:47.350 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:47 smithi037 ceph-mon[29171]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:47 smithi120 ceph-mon[32244]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:25:48.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 -- 172.21.15.37:0/900386338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc1076a0 msgr2=0x7fbcdc10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 --2- 172.21.15.37:0/900386338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc10f2b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b0d0 tx=0x7fbcd400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 -- 172.21.15.37:0/900386338 shutdown_connections 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 --2- 172.21.15.37:0/900386338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc10f2b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 --2- 172.21.15.37:0/900386338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc102d80 0x7fbcdc103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 --2- 172.21.15.37:0/900386338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc103730 0x7fbcdc106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 -- 172.21.15.37:0/900386338 >> 172.21.15.37:0/900386338 conn(0x7fbcdc078990 msgr2=0x7fbcdc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 -- 172.21.15.37:0/900386338 shutdown_connections 2024-04-26T18:25:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.931+0000 7fbce4591700 1 -- 172.21.15.37:0/900386338 wait complete. 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.933+0000 7fbce4591700 1 Processor -- start 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.933+0000 7fbce4591700 1 -- start start 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.933+0000 7fbce4591700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce232d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce232d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49508/0 (socket says 172.21.15.37:49508) 2024-04-26T18:25:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce232d700 1 -- 172.21.15.37:0/957824059 learned_addr learned my addr 172.21.15.37:0/957824059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc103730 0x7fbcdc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc112590 con 0x7fbcdc102d80 2024-04-26T18:25:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc112410 con 0x7fbcdc1076a0 2024-04-26T18:25:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc112710 con 0x7fbcdc103730 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.934+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc103730 0x7fbcdc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 -- 172.21.15.37:0/957824059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc103730 msgr2=0x7fbcdc19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc103730 0x7fbcdc19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 -- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 msgr2=0x7fbcdc199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19a550 con 0x7fbcdc102d80 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbce232d700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c9b0 tx=0x7fbcd800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.935+0000 7fbcd37fe700 1 -- 172.21.15.37:0/957824059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd801e070 con 0x7fbcdc102d80 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.936+0000 7fbcd37fe700 1 -- 172.21.15.37:0/957824059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd800ce50 con 0x7fbcdc102d80 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.936+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc19a780 con 0x7fbcdc102d80 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.936+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc112f00 con 0x7fbcdc102d80 2024-04-26T18:25:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.936+0000 7fbcd37fe700 1 -- 172.21.15.37:0/957824059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80196a0 con 0x7fbcdc102d80 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.939+0000 7fbcd37fe700 1 -- 172.21.15.37:0/957824059 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fbcd80076f0 con 0x7fbcdc102d80 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.940+0000 7fbcd37fe700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 0x7fbcc80a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.940+0000 7fbcd37fe700 1 -- 172.21.15.37:0/957824059 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fbcd80d4430 con 0x7fbcdc102d80 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 Processor -- start 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 0x7fbcc80a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 -- start start 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed86b0 0x55b28aed8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 0x55b28aed2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b28ae1c930 con 0x55b28aed3080 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b28ad82a30 con 0x55b28aed8fd0 2024-04-26T18:25:48.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbceee23b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b28ae188f0 con 0x55b28aed86b0 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 0x7fbcc80a64a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc19afd0 tx=0x7fbccc00b410 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.941+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49510/0 (socket says 172.21.15.37:49510) 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 -- 172.21.15.37:0/3911661305 learned_addr learned my addr 172.21.15.37:0/3911661305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 0x55b28aed2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce232d700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed86b0 0x55b28aed8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 -- 172.21.15.37:0/3911661305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed86b0 msgr2=0x55b28aed8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed86b0 0x55b28aed8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 -- 172.21.15.37:0/3911661305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 msgr2=0x55b28aed2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 0x55b28aed2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 -- 172.21.15.37:0/3911661305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b28ad699d0 con 0x55b28aed3080 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 0x55b28aed2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:48.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009b80 tx=0x7fbcd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41791709e3f61f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbcd0ff9700 1 -- 172.21.15.37:0/3911661305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4003c00 con 0x55b28aed3080 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.942+0000 7fbcd0ff9700 1 -- 172.21.15.37:0/3911661305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd4003da0 con 0x55b28aed3080 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbcd0ff9700 1 -- 172.21.15.37:0/3911661305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40054a0 con 0x55b28aed3080 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 -- 172.21.15.37:0/3911661305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 msgr2=0x55b28aed34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009b80 tx=0x7fbcd4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 -- 172.21.15.37:0/3911661305 shutdown_connections 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed3080 0x55b28aed34e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed86b0 0x55b28aed8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 --2- 172.21.15.37:0/3911661305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed8fd0 0x55b28aed2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.943+0000 7fbceee23b80 1 -- 172.21.15.37:0/3911661305 >> 172.21.15.37:0/3911661305 conn(0x55b28adf86c0 msgr2=0x55b28aec5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 -- 172.21.15.37:0/3911661305 shutdown_connections 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 -- 172.21.15.37:0/3911661305 wait complete. 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 -- start start 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 Processor -- start 2024-04-26T18:25:48.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28aedebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aedbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 0x55b28aedc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.944+0000 7fbceee23b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b28aedfa30 con 0x55b28aed8fd0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbceee23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b28add6b80 con 0x55b28aed86b0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbceee23b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b28aedfbb0 con 0x55b28aed3080 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbceee23b80 10 client.0 _ll_register_callbacks cb 0x55b28adb4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 0x55b28aedc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aedbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce232d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28aedebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 0x55b28aedc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49524/0 (socket says 172.21.15.37:49524) 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce2b2e700 1 -- 172.21.15.37:0/1320928518 learned_addr learned my addr 172.21.15.37:0/1320928518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 -- 172.21.15.37:0/1320928518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 msgr2=0x55b28aedebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28aedebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 -- 172.21.15.37:0/1320928518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 msgr2=0x55b28aedc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 0x55b28aedc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 -- 172.21.15.37:0/1320928518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28adda580 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce232d700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28aedebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aedbf80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc10f780 tx=0x7fbccc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43275c788db6820 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.945+0000 7fbcc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc004830 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd4003da0 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fbccc0049d0 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbccc004bd0 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 10 client.25127 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b28aec72e0 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd4003da0 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbccc02f700 con 0x55b28aed86b0 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 10 client.25127 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 10 client.25127 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:48.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 20 client.25127 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aec9870 0x55b28aecbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 4 client.25127 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.946+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b28ae2a310 con 0x55b28aec9870 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.947+0000 7fbce232d700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aec9870 0x55b28aecbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.947+0000 7fbcc5ffb700 10 client.25127 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.947+0000 7fbce232d700 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aec9870 0x55b28aecbcf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d784980a2624c018 server_cookie=4dba3ad243efcd07 in_seq=0 out_seq=0 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/1320928518 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b28ae2a310 con 0x55b28aec9870 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbcc5ffb700 10 client.25127 handle_command_reply: tid=0 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 client.25127 shutdown 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 msgr2=0x55b28aedbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aedbf80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc10f780 tx=0x7fbccc0047c0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 shutdown_connections 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aec9870 0x55b28aecbcf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed8fd0 0x55b28aedc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbcc5ffb700 0 client.25127 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28aedebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 --2- 172.21.15.37:0/1320928518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aedbf80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 >> 172.21.15.37:0/1320928518 conn(0x55b28adf86c0 msgr2=0x55b28aed1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 shutdown_connections 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 1 -- 172.21.15.37:0/1320928518 wait complete. 2024-04-26T18:25:48.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.948+0000 7fbceee23b80 20 client.25127 trim_cache size 0 max 16384 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 Processor -- start 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 -- start start 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aec9870 0x55b28aeef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed3080 0x55b28aeef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1ae280 con 0x55b28aed86b0 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1abb10 con 0x55b28aed3080 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbceee23b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1a9960 con 0x55b28aec9870 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbce2b2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49538/0 (socket says 172.21.15.37:49538) 2024-04-26T18:25:48.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbce2b2e700 1 -- 172.21.15.37:0/4093889419 learned_addr learned my addr 172.21.15.37:0/4093889419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.969+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed3080 0x55b28aeef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce232d700 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aec9870 0x55b28aeef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 -- 172.21.15.37:0/4093889419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aec9870 msgr2=0x55b28aeef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aec9870 0x55b28aeef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 -- 172.21.15.37:0/4093889419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed3080 msgr2=0x55b28aeef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed3080 0x55b28aeef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 -- 172.21.15.37:0/4093889419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b28aec50a0 con 0x55b28aed86b0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009cd0 tx=0x7fbcd40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def59986209b269 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbcc7fff700 1 -- 172.21.15.37:0/4093889419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4003cb0 con 0x55b28aed86b0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbcc7fff700 1 -- 172.21.15.37:0/4093889419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd4007950 con 0x55b28aed86b0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.970+0000 7fbcc7fff700 1 -- 172.21.15.37:0/4093889419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4005680 con 0x55b28aed86b0 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 -- 172.21.15.37:0/4093889419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 msgr2=0x55b28aedaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009cd0 tx=0x7fbcd40099e0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 -- 172.21.15.37:0/4093889419 shutdown_connections 2024-04-26T18:25:48.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aed86b0 0x55b28aedaf80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aec9870 0x55b28aeef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 --2- 172.21.15.37:0/4093889419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed3080 0x55b28aeef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 -- 172.21.15.37:0/4093889419 >> 172.21.15.37:0/4093889419 conn(0x55b28adf86c0 msgr2=0x55b28aeda870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 -- 172.21.15.37:0/4093889419 shutdown_connections 2024-04-26T18:25:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.971+0000 7fbceee23b80 1 -- 172.21.15.37:0/4093889419 wait complete. 2024-04-26T18:25:48.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 -- start start 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 Processor -- start 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28af44a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aeeeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1a9fd0 con 0x55b28aec9870 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1ab440 con 0x55b28aed86b0 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1aa6a0 con 0x55b28aed3080 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbceee23b80 10 client.0 _ll_register_callbacks cb 0x55b28adb4a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbce232d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbce232d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49550/0 (socket says 172.21.15.37:49550) 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbce232d700 1 -- 172.21.15.37:0/2544972409 learned_addr learned my addr 172.21.15.37:0/2544972409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aeeeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.972+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28af44a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 -- 172.21.15.37:0/2544972409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 msgr2=0x55b28af44a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28af44a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 -- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 msgr2=0x55b28aeeeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aeeeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 -- 172.21.15.37:0/2544972409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b28aeef290 con 0x55b28aec9870 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce2b2e700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aeeeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbce232d700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbcd809ff90 tx=0x7fbcd8007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbbb0052354123e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbcc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80a4640 con 0x55b28aec9870 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fbcd80a8a70 con 0x55b28aec9870 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd4007950 con 0x55b28aec9870 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbcd80a8f00 con 0x55b28aec9870 2024-04-26T18:25:48.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbceee23b80 10 client.15486 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.973+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b28af4b370 con 0x55b28aec9870 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8089030 con 0x55b28aec9870 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd80a42a0 con 0x55b28aec9870 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 10 client.15486 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 20 client.15486 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aed8fd0 0x55b28af4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 4 client.15486 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b2739c93", "release": true, "await": true}] 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.974+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b2739c93", "release": true, "await": true}) -- 0x7fbcd4007950 con 0x55b28aed8fd0 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.975+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aed8fd0 0x55b28af4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.975+0000 7fbcc5ffb700 10 client.15486 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:48.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.975+0000 7fbce1b2c700 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aed8fd0 0x55b28af4da80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7505502b8b912f5a server_cookie=4aa98307530dc834 in_seq=0 out_seq=0 2024-04-26T18:25:48.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.977+0000 7fbcc5ffb700 1 -- 172.21.15.37:0/2544972409 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7fbcd4007950 con 0x55b28aed8fd0 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.977+0000 7fbcc5ffb700 10 client.15486 handle_command_reply: tid=0 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.977+0000 7fbceee23b80 1 client.15486 shutdown 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 msgr2=0x55b28af444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbcd809ff90 tx=0x7fbcd8007cc0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 shutdown_connections 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b28aed8fd0 0x55b28af4da80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbcc5ffb700 0 client.15486 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b28aed86b0 0x55b28aeeeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b28aec9870 0x55b28af444c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 --2- 172.21.15.37:0/2544972409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b28aed3080 0x55b28af44a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 >> 172.21.15.37:0/2544972409 conn(0x55b28adf86c0 msgr2=0x55b28aee8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 shutdown_connections 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 1 -- 172.21.15.37:0/2544972409 wait complete. 2024-04-26T18:25:48.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.978+0000 7fbceee23b80 20 client.15486 trim_cache size 0 max 16384 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "b2739c93": { 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:25:48.982 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:48.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:48.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:48.984 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 msgr2=0x7fbcc80a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 0x7fbcc80a64a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc19afd0 tx=0x7fbccc00b410 comp rx=0 tx=0).stop 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 msgr2=0x7fbcdc19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c9b0 tx=0x7fbcd800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 shutdown_connections 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fbcc80a3fe0 0x7fbcc80a64a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbcdc1076a0 0x7fbcdc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcdc102d80 0x7fbcdc19f790 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 --2- 172.21.15.37:0/957824059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcdc103730 0x7fbcdc19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:48.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.979+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 >> 172.21.15.37:0/957824059 conn(0x7fbcdc078990 msgr2=0x7fbcdc1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:48.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.980+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 shutdown_connections 2024-04-26T18:25:48.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:48.980+0000 7fbce4591700 1 -- 172.21.15.37:0/957824059 wait complete. 2024-04-26T18:25:49.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:49 smithi037 ceph-mon[29171]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:49.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:49 smithi170 ceph-mon[32697]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:49 smithi120 ceph-mon[32244]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:49.825 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b2739c93', seconds elapsed: 42.4 2024-04-26T18:25:49.825 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:25:49.826 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:25:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:51.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:51 smithi170 ceph-mon[32697]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:51.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:51 smithi037 ceph-mon[29171]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:51.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:51 smithi120 ceph-mon[32244]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 -- 172.21.15.37:0/3340545489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300108380 msgr2=0x7f2300108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 --2- 172.21.15.37:0/3340545489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300108380 0x7f2300108760 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009990 tx=0x7f22f0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 -- 172.21.15.37:0/3340545489 shutdown_connections 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 --2- 172.21.15.37:0/3340545489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 --2- 172.21.15.37:0/3340545489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300108380 0x7f2300108760 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 --2- 172.21.15.37:0/3340545489 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 0x7f2300102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.918+0000 7f2306279700 1 -- 172.21.15.37:0/3340545489 >> 172.21.15.37:0/3340545489 conn(0x7f2300079290 msgr2=0x7f23000796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.919+0000 7f2306279700 1 -- 172.21.15.37:0/3340545489 shutdown_connections 2024-04-26T18:25:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.919+0000 7f2306279700 1 -- 172.21.15.37:0/3340545489 wait complete. 2024-04-26T18:25:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.920+0000 7f2306279700 1 Processor -- start 2024-04-26T18:25:51.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.920+0000 7f2306279700 1 -- start start 2024-04-26T18:25:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 0x7f230019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 0x7f2300199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23001135c0 con 0x7f2300199af0 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2300113440 con 0x7f2300102310 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2306279700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2300113740 con 0x7f2300102cd0 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:41334/0 (socket says 172.21.15.37:41334) 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f22fffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 0x7f230019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f22ff7fe700 1 -- 172.21.15.37:0/2553207730 learned_addr learned my addr 172.21.15.37:0/2553207730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.921+0000 7f2304816700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 0x7f2300199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 -- 172.21.15.37:0/2553207730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 msgr2=0x7f230019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 0x7f230019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 -- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 msgr2=0x7f2300199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 0x7f2300199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 -- 172.21.15.37:0/2553207730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230019a670 con 0x7f2300102cd0 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f2304816700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 0x7f2300199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22ff7fe700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b570 tx=0x7f22f400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f402a020 con 0x7f2300102cd0 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22f4009e40 con 0x7f2300102cd0 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4013070 con 0x7f2300102cd0 2024-04-26T18:25:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230010fb80 con 0x7f2300102cd0 2024-04-26T18:25:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.922+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2300110010 con 0x7f2300102cd0 2024-04-26T18:25:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.923+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2300103a30 con 0x7f2300102cd0 2024-04-26T18:25:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.925+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f22f4004a30 con 0x7f2300102cd0 2024-04-26T18:25:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.926+0000 7f22fd7fa700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 0x7f22e80a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.926+0000 7f22fffff700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 0x7f22e80a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.926+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f22f40e2020 con 0x7f2300102cd0 2024-04-26T18:25:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.926+0000 7f22fffff700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 0x7f22e80a6520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22f00059d0 tx=0x7f22f0005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:51.929+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22f4034020 con 0x7f2300102cd0 2024-04-26T18:25:52.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.091+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f22f40aa470 con 0x7f2300102cd0 2024-04-26T18:25:52.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.221+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f230002e490 con 0x7f2300102cd0 2024-04-26T18:25:52.222 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:25:52.222 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:25:52.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.222+0000 7f22fd7fa700 1 -- 172.21.15.37:0/2553207730 <== mon.1 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f22f40d9090 con 0x7f2300102cd0 2024-04-26T18:25:52.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.224+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 msgr2=0x7f22e80a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.224+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 0x7f22e80a6520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22f00059d0 tx=0x7f22f0005960 comp rx=0 tx=0).stop 2024-04-26T18:25:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.224+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 msgr2=0x7f230019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.224+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22f400b570 tx=0x7f22f400b880 comp rx=0 tx=0).stop 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 shutdown_connections 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2300199af0 0x7f2300199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f22e80a4060 0x7f22e80a6520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2300102310 0x7f230019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 --2- 172.21.15.37:0/2553207730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2300102cd0 0x7f230019fe70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 >> 172.21.15.37:0/2553207730 conn(0x7f2300079290 msgr2=0x7f23001013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 shutdown_connections 2024-04-26T18:25:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:52.225+0000 7f2306279700 1 -- 172.21.15.37:0/2553207730 wait complete. 2024-04-26T18:25:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:25:53.044 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:25:53.044 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:25:53.323 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:53.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:53 smithi037 ceph-mon[29171]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T18:25:53.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:53 smithi037 ceph-mon[29171]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-26T18:25:53.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:53 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2553207730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:53 smithi120 ceph-mon[32244]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T18:25:53.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:53 smithi120 ceph-mon[32244]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-26T18:25:53.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:53 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2553207730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:53 smithi170 ceph-mon[32697]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T18:25:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:53 smithi170 ceph-mon[32697]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-26T18:25:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:53 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2553207730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 -- 172.21.15.37:0/4201329083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558101740 msgr2=0x7f0558101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 --2- 172.21.15.37:0/4201329083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558101740 0x7f0558101b20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0554009900 tx=0x7f055402a410 comp rx=0 tx=0).stop 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 -- 172.21.15.37:0/4201329083 shutdown_connections 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 --2- 172.21.15.37:0/4201329083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055810f2f0 0x7f05581116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 --2- 172.21.15.37:0/4201329083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0558102060 0x7f055810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 --2- 172.21.15.37:0/4201329083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558101740 0x7f0558101b20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.299+0000 7f055fed1640 1 -- 172.21.15.37:0/4201329083 >> 172.21.15.37:0/4201329083 conn(0x7f05580789b0 msgr2=0x7f0558078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- 172.21.15.37:0/4201329083 shutdown_connections 2024-04-26T18:25:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- 172.21.15.37:0/4201329083 wait complete. 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 Processor -- start 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- start start 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f055810f2f0 0x7f055819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 0x7f05581a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0558114dd0 con 0x7f0558102060 2024-04-26T18:25:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0558114c50 con 0x7f055819fd80 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.300+0000 7f055fed1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0558114f50 con 0x7f055810f2f0 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49586/0 (socket says 172.21.15.37:49586) 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 -- 172.21.15.37:0/302285231 learned_addr learned my addr 172.21.15.37:0/302285231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055d46c640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f055810f2f0 0x7f055819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055e46e640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 0x7f05581a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 -- 172.21.15.37:0/302285231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f055810f2f0 msgr2=0x7f055819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f055810f2f0 0x7f055819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 -- 172.21.15.37:0/302285231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 msgr2=0x7f05581a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 0x7f05581a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 -- 172.21.15.37:0/302285231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05581a0870 con 0x7f0558102060 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055e46e640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 0x7f05581a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f055dc6d640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0554006010 tx=0x7f0554004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.301+0000 7f054effd640 1 -- 172.21.15.37:0/302285231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055403f030 con 0x7f0558102060 2024-04-26T18:25:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.302+0000 7f054effd640 1 -- 172.21.15.37:0/302285231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0554004440 con 0x7f0558102060 2024-04-26T18:25:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.302+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05581a0b00 con 0x7f0558102060 2024-04-26T18:25:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.302+0000 7f054effd640 1 -- 172.21.15.37:0/302285231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554049060 con 0x7f0558102060 2024-04-26T18:25:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.302+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05581a8270 con 0x7f0558102060 2024-04-26T18:25:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.305+0000 7f054effd640 1 -- 172.21.15.37:0/302285231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0554042030 con 0x7f0558102060 2024-04-26T18:25:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.305+0000 7f0562bed740 1 Processor -- start 2024-04-26T18:25:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.305+0000 7f0562bed740 1 -- start start 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f054effd640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 0x7f05440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f054effd640 1 -- 172.21.15.37:0/302285231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05540f0e20 con 0x7f0558102060 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f055e46e640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 0x7f05440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590a7124460 con 0x5590a71c90e0 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5590a70de590 con 0x5590a71bdf00 2024-04-26T18:25:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.306+0000 7f0562bed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590a7120420 con 0x5590a71be820 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055e46e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:41362/0 (socket says 172.21.15.37:41362) 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 -- 172.21.15.37:0/483841357 learned_addr learned my addr 172.21.15.37:0/483841357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055dc6d640 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 -- 172.21.15.37:0/483841357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 msgr2=0x5590a71be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 -- 172.21.15.37:0/483841357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 msgr2=0x5590a71cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 -- 172.21.15.37:0/483841357 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a70eb0d0 con 0x5590a71be820 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055e46e640 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.307+0000 7f055d46c640 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0548009ca0 tx=0x7f0548011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4235cc4cab1a432 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f055e46e640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 0x7f05440a6280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f055806c110 tx=0x7f0550014800 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f0543fff640 1 -- 172.21.15.37:0/483841357 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548005220 con 0x5590a71be820 2024-04-26T18:25:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f0543fff640 1 -- 172.21.15.37:0/483841357 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0548004c50 con 0x5590a71be820 2024-04-26T18:25:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f0543fff640 1 -- 172.21.15.37:0/483841357 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548019650 con 0x5590a71be820 2024-04-26T18:25:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f0562bed740 1 -- 172.21.15.37:0/483841357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 msgr2=0x5590a71c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.308+0000 7f0562bed740 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0548009ca0 tx=0x7f0548011810 comp rx=0 tx=0).stop 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 -- 172.21.15.37:0/483841357 shutdown_connections 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71be820 0x5590a71c8ba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 --2- 172.21.15.37:0/483841357 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 -- 172.21.15.37:0/483841357 >> 172.21.15.37:0/483841357 conn(0x5590a71ba410 msgr2=0x5590a71ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 -- 172.21.15.37:0/483841357 shutdown_connections 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 -- 172.21.15.37:0/483841357 wait complete. 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 -- start start 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.309+0000 7f0562bed740 1 Processor -- start 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 0x5590a71c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590a71cbd10 con 0x5590a71c90e0 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5590a70abb60 con 0x5590a71be820 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590a71cbe90 con 0x5590a71bdf00 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f0562bed740 10 client.0 _ll_register_callbacks cb 0x5590a70d02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:59624/0 (socket says 172.21.15.37:59624) 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 -- 172.21.15.37:0/2390551444 learned_addr learned my addr 172.21.15.37:0/2390551444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055e46e640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055dc6d640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 0x5590a71c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 -- 172.21.15.37:0/2390551444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 msgr2=0x5590a71c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 0x5590a71c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 -- 172.21.15.37:0/2390551444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 msgr2=0x5590a71c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 -- 172.21.15.37:0/2390551444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a6e36210 con 0x5590a71be820 2024-04-26T18:25:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055dc6d640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 0x5590a71c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055e46e640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.310+0000 7f055d46c640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0548005230 tx=0x7f0548005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66122608cab9997 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f05417fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548029070 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0548023840 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0548004c50 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0548035050 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f0562bed740 10 client.25151 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590a6f02620 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.311+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0548004c50 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0548019850 con 0x5590a71be820 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 10 client.25151 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 10 client.25151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 20 client.25151 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71d8f50 0x5590a71db340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 4 client.25151 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a7132070 con 0x5590a71d8f50 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.312+0000 7f055dc6d640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71d8f50 0x5590a71db340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.313+0000 7f05417fa640 10 client.25151 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.313+0000 7f055dc6d640 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71d8f50 0x5590a71db340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec7646eae0aa5f8 server_cookie=7193641040d11b04 in_seq=0 out_seq=0 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.313+0000 7f05417fa640 1 -- 172.21.15.37:0/2390551444 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590a7132070 con 0x5590a71d8f50 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.313+0000 7f05417fa640 10 client.25151 handle_command_reply: tid=0 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.313+0000 7f0562bed740 1 client.25151 shutdown 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 msgr2=0x5590a71c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0548005230 tx=0x7f0548005260 comp rx=0 tx=0).stop 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 shutdown_connections 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71d8f50 0x5590a71db340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71be820 0x5590a71c6600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f05417fa640 0 client.25151 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71c90e0 0x5590a71c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 --2- 172.21.15.37:0/2390551444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71bdf00 0x5590a71c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 >> 172.21.15.37:0/2390551444 conn(0x5590a71ba410 msgr2=0x5590a71c0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 shutdown_connections 2024-04-26T18:25:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 1 -- 172.21.15.37:0/2390551444 wait complete. 2024-04-26T18:25:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.314+0000 7f0562bed740 20 client.25151 trim_cache size 0 max 16384 2024-04-26T18:25:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 Processor -- start 2024-04-26T18:25:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 -- start start 2024-04-26T18:25:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05581b3560 con 0x5590a71be820 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05581b0e30 con 0x5590a71bdf00 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f0562bed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05581aec80 con 0x5590a71ba410 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055d46c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49622/0 (socket says 172.21.15.37:49622) 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 -- 172.21.15.37:0/1194454656 learned_addr learned my addr 172.21.15.37:0/1194454656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055dc6d640 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 -- 172.21.15.37:0/1194454656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 msgr2=0x5590a71d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71d4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 -- 172.21.15.37:0/1194454656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 msgr2=0x5590a71b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.345+0000 7f055e46e640 1 -- 172.21.15.37:0/1194454656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a71c5070 con 0x5590a71be820 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.346+0000 7f055d46c640 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:54.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.346+0000 7f055e46e640 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05500148f0 tx=0x7f0550009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdec1ebf5a0838b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.346+0000 7f05437fe640 1 -- 172.21.15.37:0/1194454656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550014920 con 0x5590a71be820 2024-04-26T18:25:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.346+0000 7f05437fe640 1 -- 172.21.15.37:0/1194454656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0550014130 con 0x5590a71be820 2024-04-26T18:25:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.346+0000 7f05437fe640 1 -- 172.21.15.37:0/1194454656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550022460 con 0x5590a71be820 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- 172.21.15.37:0/1194454656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 msgr2=0x5590a71c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05500148f0 tx=0x7f0550009500 comp rx=0 tx=0).stop 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- 172.21.15.37:0/1194454656 shutdown_connections 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c6960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 --2- 172.21.15.37:0/1194454656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- 172.21.15.37:0/1194454656 >> 172.21.15.37:0/1194454656 conn(0x5590a71db6b0 msgr2=0x5590a71ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- 172.21.15.37:0/1194454656 shutdown_connections 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- 172.21.15.37:0/1194454656 wait complete. 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 -- start start 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.347+0000 7f0562bed740 1 Processor -- start 2024-04-26T18:25:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05581af2f0 con 0x5590a71be820 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05581b0760 con 0x5590a71bdf00 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05581af9c0 con 0x5590a71ba410 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f0562bed740 10 client.0 _ll_register_callbacks cb 0x7f05500146d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055e46e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055d46c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:41394/0 (socket says 172.21.15.37:41394) 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 -- 172.21.15.37:0/1787740614 learned_addr learned my addr 172.21.15.37:0/1787740614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 -- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 msgr2=0x5590a71c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.348+0000 7f055dc6d640 1 -- 172.21.15.37:0/1787740614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 msgr2=0x5590a71c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f055dc6d640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f055dc6d640 1 -- 172.21.15.37:0/1787740614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a723e5b0 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f055e46e640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f055dc6d640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05540eeb60 tx=0x7f055403b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661d390678a6178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f05417fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540d6070 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05540ee7e0 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0550014130 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0554035040 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f0562bed740 10 client.25114 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5590a723f900 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.349+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05540c1160 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f05540a7040 con 0x5590a71ba410 2024-04-26T18:25:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 10 client.25114 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 10 client.25114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 20 client.25114 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71c90e0 0x5590a7242030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 4 client.25114 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0550014130 con 0x5590a71c90e0 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f055d46c640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71c90e0 0x5590a7242030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f055d46c640 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71c90e0 0x5590a7242030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd243038b125eaae server_cookie=7987d10d213e1b4f in_seq=0 out_seq=0 2024-04-26T18:25:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.350+0000 7f05417fa640 10 client.25114 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f05417fa640 1 -- 172.21.15.37:0/1787740614 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0550014130 con 0x5590a71c90e0 2024-04-26T18:25:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f05417fa640 10 client.25114 handle_command_reply: tid=0 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 client.25114 shutdown 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 msgr2=0x5590a71c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05540eeb60 tx=0x7f055403b460 comp rx=0 tx=0).stop 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 shutdown_connections 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5590a71c90e0 0x5590a7242030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5590a71bdf00 0x5590a71c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590a71be820 0x5590a71c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f05417fa640 0 client.25114 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 --2- 172.21.15.37:0/1787740614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a71ba410 0x5590a71c3000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 >> 172.21.15.37:0/1787740614 conn(0x5590a71db6b0 msgr2=0x5590a71b95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 shutdown_connections 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 1 -- 172.21.15.37:0/1787740614 wait complete. 2024-04-26T18:25:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.356+0000 7f0562bed740 20 client.25114 trim_cache size 0 max 16384 2024-04-26T18:25:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:25:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:25:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:25:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 msgr2=0x7f05440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 0x7f05440a6280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f055806c110 tx=0x7f0550014800 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 msgr2=0x7f055819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0554006010 tx=0x7f0554004040 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 shutdown_connections 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05440a3dc0 0x7f05440a6280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f055819fd80 0x7f05581a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f055810f2f0 0x7f055819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 --2- 172.21.15.37:0/302285231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0558102060 0x7f055819b4c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 >> 172.21.15.37:0/302285231 conn(0x7f05580789b0 msgr2=0x7f0558110a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 shutdown_connections 2024-04-26T18:25:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:25:54.357+0000 7f055fed1640 1 -- 172.21.15.37:0/302285231 wait complete. 2024-04-26T18:25:55.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 -- 172.21.15.37:0/2530502533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 msgr2=0x7f8b0c06b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/2530502533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c06b5a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009990 tx=0x7f8afc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 -- 172.21.15.37:0/2530502533 shutdown_connections 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/2530502533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c06bae0 0x7f8b0c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/2530502533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c1094d0 0x7f8b0c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/2530502533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c06b5a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 -- 172.21.15.37:0/2530502533 >> 172.21.15.37:0/2530502533 conn(0x7f8b0c0792c0 msgr2=0x7f8b0c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 -- 172.21.15.37:0/2530502533 shutdown_connections 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.059+0000 7f8b13cd3700 1 -- 172.21.15.37:0/2530502533 wait complete. 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.060+0000 7f8b13cd3700 1 Processor -- start 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.060+0000 7f8b13cd3700 1 -- start start 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c06bae0 0x7f8b0c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1094d0 0x7f8b0c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1135d0 con 0x7f8b0c06b120 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c113450 con 0x7f8b0c06bae0 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b13cd3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c113750 con 0x7f8b0c1094d0 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43488/0 (socket says 172.21.15.37:43488) 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 -- 172.21.15.37:0/3821228154 learned_addr learned my addr 172.21.15.37:0/3821228154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c06bae0 0x7f8b0c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 -- 172.21.15.37:0/3821228154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1094d0 msgr2=0x7f8b0c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1094d0 0x7f8b0c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 -- 172.21.15.37:0/3821228154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c06bae0 msgr2=0x7f8b0c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c06bae0 0x7f8b0c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.061+0000 7f8b11a6f700 1 -- 172.21.15.37:0/3821228154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c1a7ef0 con 0x7f8b0c06b120 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.062+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800e9a0 tx=0x7f8b0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.062+0000 7f8b02ffd700 1 -- 172.21.15.37:0/3821228154 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0800cbd0 con 0x7f8b0c06b120 2024-04-26T18:25:55.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.062+0000 7f8b02ffd700 1 -- 172.21.15.37:0/3821228154 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b0800cd70 con 0x7f8b0c06b120 2024-04-26T18:25:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.062+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c1a81e0 con 0x7f8b0c06b120 2024-04-26T18:25:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.063+0000 7f8b02ffd700 1 -- 172.21.15.37:0/3821228154 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08018660 con 0x7f8b0c06b120 2024-04-26T18:25:55.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.063+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a86f0 con 0x7f8b0c06b120 2024-04-26T18:25:55.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.066+0000 7f8b02ffd700 1 -- 172.21.15.37:0/3821228154 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b08018860 con 0x7f8b0c06b120 2024-04-26T18:25:55.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.067+0000 7f8b02ffd700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 0x7f8af80a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.067+0000 7f8b02ffd700 1 -- 172.21.15.37:0/3821228154 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8b080d08f0 con 0x7f8b0c06b120 2024-04-26T18:25:55.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.067+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 0x7f8af80a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.068+0000 7f8b1e565b80 1 Processor -- start 2024-04-26T18:25:55.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.068+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 0x7f8af80a6470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0098e0 tx=0x7f8afc005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.068+0000 7f8b1e565b80 1 -- start start 2024-04-26T18:25:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 0x55a2565cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43504/0 (socket says 172.21.15.37:43504) 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 0x55a2565cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 0x55a2565cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:37760/0 (socket says 172.21.15.37:37760) 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1126e700 1 -- 172.21.15.37:0/581139226 learned_addr learned my addr 172.21.15.37:0/581139226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a25651b9c0 con 0x55a2565cf850 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.069+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a256481ac0 con 0x55a2565cef30 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a256517980 con 0x55a2565c9900 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 -- 172.21.15.37:0/581139226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 msgr2=0x55a2565cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 0x55a2565cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 -- 172.21.15.37:0/581139226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 msgr2=0x55a2565c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b12270700 1 -- 172.21.15.37:0/581139226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a256468a60 con 0x55a2565c9900 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.070+0000 7f8b1126e700 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.071+0000 7f8b12270700 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c9d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b04008fa0 tx=0x7f8b0400d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a378f0de244d6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.071+0000 7f8af7fff700 1 -- 172.21.15.37:0/581139226 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400de60 con 0x55a2565c9900 2024-04-26T18:25:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.071+0000 7f8af7fff700 1 -- 172.21.15.37:0/581139226 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b04018070 con 0x55a2565c9900 2024-04-26T18:25:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.071+0000 7f8af7fff700 1 -- 172.21.15.37:0/581139226 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04012740 con 0x55a2565c9900 2024-04-26T18:25:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 msgr2=0x55a2565c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c9d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8b04008fa0 tx=0x7f8b0400d3c0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 shutdown_connections 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c9d60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cef30 0x55a2565cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/581139226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cf850 0x55a2565c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 >> 172.21.15.37:0/581139226 conn(0x55a2564f7750 msgr2=0x55a2565c4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 shutdown_connections 2024-04-26T18:25:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.072+0000 7f8b1e565b80 1 -- 172.21.15.37:0/581139226 wait complete. 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 -- start start 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 Processor -- start 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 0x55a2565d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2565d62c0 con 0x55a2565c9900 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a2564d5c10 con 0x55a2565cf850 2024-04-26T18:25:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.073+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a2565d6440 con 0x55a2565cef30 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1e565b80 10 client.0 _ll_register_callbacks cb 0x55a2564b3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:52720/0 (socket says 172.21.15.37:52720) 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 -- 172.21.15.37:0/3844519049 learned_addr learned my addr 172.21.15.37:0/3844519049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 0x55a2565d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b12270700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 -- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 msgr2=0x55a2565d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 -- 172.21.15.37:0/3844519049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 msgr2=0x55a2565d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 0x55a2565d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 -- 172.21.15.37:0/3844519049 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2564d9610 con 0x55a2565cef30 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 0x55a2565d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b12270700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8b1126e700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00be80 tx=0x7f8afc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459130a067db3c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.074+0000 7f8af57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:25:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc005690 con 0x55a2565cef30 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8afc00f460 con 0x55a2565cef30 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b04018070 con 0x55a2565cef30 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8afc00f660 con 0x55a2565cef30 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 10 client.25120 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2565e92d0 con 0x55a2565cef30 2024-04-26T18:25:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b04018070 con 0x55a2565cef30 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8afc031840 con 0x55a2565cef30 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 10 client.25120 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 10 client.25120 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 20 client.25120 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565eb960 0x55a2565edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 4 client.25120 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.075+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2565293a0 con 0x55a2565eb960 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.076+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565eb960 0x55a2565edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.076+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565eb960 0x55a2565edd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3f955c29d35618 server_cookie=12db37a5a0d81b55 in_seq=0 out_seq=0 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.076+0000 7f8af57fa700 10 client.25120 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8af57fa700 1 -- 172.21.15.37:0/3844519049 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2565293a0 con 0x55a2565eb960 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8af57fa700 10 client.25120 handle_command_reply: tid=0 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 client.25120 shutdown 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 msgr2=0x55a2565d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00be80 tx=0x7f8afc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 shutdown_connections 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565eb960 0x55a2565edd50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565c9900 0x55a2565d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/3844519049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565cef30 0x55a2565d2830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8af57fa700 0 client.25120 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 >> 172.21.15.37:0/3844519049 conn(0x55a2564f7750 msgr2=0x55a2565c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 shutdown_connections 2024-04-26T18:25:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 1 -- 172.21.15.37:0/3844519049 wait complete. 2024-04-26T18:25:55.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.077+0000 7f8b1e565b80 20 client.25120 trim_cache size 0 max 16384 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 Processor -- start 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 -- start start 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c0727f0 con 0x55a2565cef30 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c070080 con 0x55a2565cf850 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c19d990 con 0x55a2565c9900 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43522/0 (socket says 172.21.15.37:43522) 2024-04-26T18:25:55.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1126e700 1 -- 172.21.15.37:0/1416791939 learned_addr learned my addr 172.21.15.37:0/1416791939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b12270700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b1126e700 1 -- 172.21.15.37:0/1416791939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 msgr2=0x55a2565c92e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.098+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b1126e700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b1126e700 1 -- 172.21.15.37:0/1416791939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 msgr2=0x55a2565c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b1126e700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b1126e700 1 -- 172.21.15.37:0/1416791939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2565d5c90 con 0x55a2565cef30 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b12270700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:55.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.099+0000 7f8b1126e700 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8afc031d00 tx=0x7f8afc00b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83c48487a6cb837 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.100+0000 7f8af77fe700 1 -- 172.21.15.37:0/1416791939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0175f0 con 0x55a2565cef30 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.100+0000 7f8af77fe700 1 -- 172.21.15.37:0/1416791939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc017de0 con 0x55a2565cef30 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.100+0000 7f8af77fe700 1 -- 172.21.15.37:0/1416791939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc031430 con 0x55a2565cef30 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- 172.21.15.37:0/1416791939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 msgr2=0x55a2565df1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8afc031d00 tx=0x7f8afc00b2c0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- 172.21.15.37:0/1416791939 shutdown_connections 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a2565c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/1416791939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a2565df1f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- 172.21.15.37:0/1416791939 >> 172.21.15.37:0/1416791939 conn(0x55a2564f7750 msgr2=0x55a2565bd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- 172.21.15.37:0/1416791939 shutdown_connections 2024-04-26T18:25:55.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- 172.21.15.37:0/1416791939 wait complete. 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 -- start start 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.101+0000 7f8b1e565b80 1 Processor -- start 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a25664d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06e540 con 0x55a2565cef30 2024-04-26T18:25:55.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06f9b0 con 0x55a2565cf850 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c06ec10 con 0x55a2565c9900 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b11a6f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:52736/0 (socket says 172.21.15.37:52736) 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b11a6f700 1 -- 172.21.15.37:0/2233834443 learned_addr learned my addr 172.21.15.37:0/2233834443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1126e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43536/0 (socket says 172.21.15.37:43536) 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b1e565b80 10 client.0 _ll_register_callbacks cb 0x55a2564b3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.102+0000 7f8b12270700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a25664d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 -- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 msgr2=0x55a25664d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a25664d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 -- 172.21.15.37:0/2233834443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 msgr2=0x55a25664d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 -- 172.21.15.37:0/2233834443 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2565c0880 con 0x55a2565c9900 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b1126e700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:25:55.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b11a6f700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b08099250 tx=0x7f8b0801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3185a291b67e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8af57fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:25:55.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b080a5260 con 0x55a2565c9900 2024-04-26T18:25:55.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8afc017de0 con 0x55a2565c9900 2024-04-26T18:25:55.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f8b0809dbb0 con 0x55a2565c9900 2024-04-26T18:25:55.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.103+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b0809ddb0 con 0x55a2565c9900 2024-04-26T18:25:55.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 10 client.25123 fetch_fsmap learned FSMap version 26 2024-04-26T18:25:55.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2565da7c0 con 0x55a2565c9900 2024-04-26T18:25:55.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc017de0 con 0x55a2565c9900 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8b080986f0 con 0x55a2565c9900 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 10 client.25123 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 10 client.25123 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 20 client.25123 populate_metadata read hostname 'smithi037' 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565dae90 0x55a2565e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 4 client.25123 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.104+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55a256523440 con 0x55a2565dae90 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.105+0000 7f8b1126e700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565dae90 0x55a2565e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.105+0000 7f8b1126e700 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565dae90 0x55a2565e83f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394ad2f45994d63e server_cookie=5b1f2c4fcab12331 in_seq=0 out_seq=0 2024-04-26T18:25:55.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.105+0000 7f8af57fa700 10 client.25123 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8af57fa700 1 -- 172.21.15.37:0/2233834443 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a256523440 con 0x55a2565dae90 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8af57fa700 10 client.25123 handle_command_reply: tid=0 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8b1e565b80 1 client.25123 shutdown 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 msgr2=0x55a2565dbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8b08099250 tx=0x7f8b0801a040 comp rx=0 tx=0).stop 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 shutdown_connections 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.113+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a2565dae90 0x55a2565e83f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8af57fa700 0 client.25123 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:25:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2565cf850 0x55a25664d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a2565c9900 0x55a2565dbb50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 --2- 172.21.15.37:0/2233834443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2565cef30 0x55a25664d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 >> 172.21.15.37:0/2233834443 conn(0x55a2564f7750 msgr2=0x55a2565bdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 shutdown_connections 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 1 -- 172.21.15.37:0/2233834443 wait complete. 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T18:25:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "33c0af61": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:25:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.114+0000 7f8b1e565b80 20 client.25123 trim_cache size 0 max 16384 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 msgr2=0x7f8af80a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 0x7f8af80a6470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0098e0 tx=0x7f8afc005960 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 msgr2=0x7f8b0c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800e9a0 tx=0x7f8b0800ecb0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 shutdown_connections 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1094d0 0x7f8b0c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b0c06b120 0x7f8b0c1023f0 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800e9a0 tx=0x7f8b0800ecb0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8af80a3fb0 0x7f8af80a6470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 --2- 172.21.15.37:0/3821228154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b0c06bae0 0x7f8b0c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 >> 172.21.15.37:0/3821228154 conn(0x7f8b0c0792c0 msgr2=0x7f8b0c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 shutdown_connections 2024-04-26T18:25:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:25:55.115+0000 7f8b13cd3700 1 -- 172.21.15.37:0/3821228154 wait complete. 2024-04-26T18:25:55.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:55 smithi037 ceph-mon[29171]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:25:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:55 smithi120 ceph-mon[32244]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:25:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:55 smithi170 ceph-mon[32697]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:25:55.815 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '33c0af61', quiesce took 6.0 seconds. Will release after: 30.0 2024-04-26T18:25:57.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:57 smithi037 ceph-mon[29171]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T18:25:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:57 smithi120 ceph-mon[32244]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T18:25:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:57 smithi170 ceph-mon[32697]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T18:25:59.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:25:59 smithi037 ceph-mon[29171]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:25:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:25:59 smithi120 ceph-mon[32244]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:25:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:25:59 smithi170 ceph-mon[32697]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:26:01.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:01 smithi037 ceph-mon[29171]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:26:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:01 smithi120 ceph-mon[32244]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:26:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:01 smithi170 ceph-mon[32697]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T18:26:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:03 smithi037 ceph-mon[29171]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-26T18:26:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:03 smithi120 ceph-mon[32244]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-26T18:26:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:03 smithi170 ceph-mon[32697]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-26T18:26:05.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:05 smithi037 ceph-mon[29171]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-26T18:26:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:05 smithi120 ceph-mon[32244]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-26T18:26:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:05 smithi170 ceph-mon[32697]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 9 op/s 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 -- 172.21.15.37:0/4048124878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8101860 msgr2=0x7fe0f8101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 --2- 172.21.15.37:0/4048124878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8101860 0x7fe0f8101ce0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009930 tx=0x7fe0ec02a420 comp rx=0 tx=0).stop 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 -- 172.21.15.37:0/4048124878 shutdown_connections 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 --2- 172.21.15.37:0/4048124878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8102220 0x7fe0f810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 --2- 172.21.15.37:0/4048124878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8101860 0x7fe0f8101ce0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 --2- 172.21.15.37:0/4048124878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 -- 172.21.15.37:0/4048124878 >> 172.21.15.37:0/4048124878 conn(0x7fe0f80789b0 msgr2=0x7fe0f8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 -- 172.21.15.37:0/4048124878 shutdown_connections 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.452+0000 7fe0fde46640 1 -- 172.21.15.37:0/4048124878 wait complete. 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 Processor -- start 2024-04-26T18:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 -- start start 2024-04-26T18:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101860 0x7fe0f8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8111ae0 con 0x7fe0f8102220 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8111960 con 0x7fe0f8107820 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0fde46640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8111c60 con 0x7fe0f8101860 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39642/0 (socket says 172.21.15.37:39642) 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 -- 172.21.15.37:0/891766275 learned_addr learned my addr 172.21.15.37:0/891766275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101860 0x7fe0f8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 -- 172.21.15.37:0/891766275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101860 msgr2=0x7fe0f8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101860 0x7fe0f8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 -- 172.21.15.37:0/891766275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 msgr2=0x7fe0f819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f6ffd640 1 -- 172.21.15.37:0/891766275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f819e130 con 0x7fe0f8102220 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.453+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec004770 tx=0x7fe0ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0f4ff9640 1 -- 172.21.15.37:0/891766275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec03f070 con 0x7fe0f8102220 2024-04-26T18:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0f4ff9640 1 -- 172.21.15.37:0/891766275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec03b460 con 0x7fe0f8102220 2024-04-26T18:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f819e3c0 con 0x7fe0f8102220 2024-04-26T18:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0f4ff9640 1 -- 172.21.15.37:0/891766275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec035070 con 0x7fe0f8102220 2024-04-26T18:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.454+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f81a5cb0 con 0x7fe0f8102220 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.456+0000 7fe100c33740 1 Processor -- start 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.456+0000 7fe100c33740 1 -- start start 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad535eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad53614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560ad52ba460 con 0x560ad5353f00 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560ad5274590 con 0x560ad535f0e0 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe100c33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560ad52b6420 con 0x560ad5354820 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39656/0 (socket says 172.21.15.37:39656) 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 -- 172.21.15.37:0/277614387 learned_addr learned my addr 172.21.15.37:0/277614387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f4ff9640 1 -- 172.21.15.37:0/891766275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0ec03b8d0 con 0x7fe0f8102220 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad535eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 -- 172.21.15.37:0/277614387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 msgr2=0x560ad535eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad535eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 -- 172.21.15.37:0/277614387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 msgr2=0x560ad53614d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad53614d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 -- 172.21.15.37:0/277614387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad52810d0 con 0x560ad5353f00 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009ea0 tx=0x7fe0f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30a7b977e48f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0ddffb640 1 -- 172.21.15.37:0/277614387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004d20 con 0x560ad5353f00 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0ddffb640 1 -- 172.21.15.37:0/277614387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00054d0 con 0x560ad5353f00 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.457+0000 7fe0ddffb640 1 -- 172.21.15.37:0/277614387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00117a0 con 0x560ad5353f00 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe0f4ff9640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 0x7fe0e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 0x7fe0e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe0f4ff9640 1 -- 172.21.15.37:0/891766275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe0ec0f0c00 con 0x7fe0f8102220 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 -- 172.21.15.37:0/277614387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 msgr2=0x560ad53542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009ea0 tx=0x7fe0f0009f90 comp rx=0 tx=0).stop 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 0x7fe0e00a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8005fd0 tx=0x7fe0e800b5d0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 -- 172.21.15.37:0/277614387 shutdown_connections 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad53614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad535eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 --2- 172.21.15.37:0/277614387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad53542e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 -- 172.21.15.37:0/277614387 >> 172.21.15.37:0/277614387 conn(0x560ad5350410 msgr2=0x560ad53507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 -- 172.21.15.37:0/277614387 shutdown_connections 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.458+0000 7fe100c33740 1 -- 172.21.15.37:0/277614387 wait complete. 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 -- start start 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 Processor -- start 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad535c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad535caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560ad5361d10 con 0x560ad5353f00 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560ad5241b60 con 0x560ad5354820 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560ad5361e90 con 0x560ad535f0e0 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe100c33740 10 client.0 _ll_register_callbacks cb 0x560ad52662b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39672/0 (socket says 172.21.15.37:39672) 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1624865873 learned_addr learned my addr 172.21.15.37:0/1624865873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad535caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.459+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad535c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1624865873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 msgr2=0x560ad535caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad535caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 msgr2=0x560ad535c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad535c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1624865873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad4fcc210 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8012730 tx=0x7fe0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21d3e9bd380a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e801f920 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe0e801fac0 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f00054d0 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0e801fcc0 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe100c33740 10 client.15540 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ad5098620 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.460+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00054d0 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0e802f450 con 0x560ad5353f00 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 20 client.15540 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad536f280 0x560ad5371670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 4 client.15540 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad52c8070 con 0x560ad536f280 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad536f280 0x560ad5371670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad536f280 0x560ad5371670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c293bc4dd8136efa server_cookie=62abcc5aef3d0f1d in_seq=0 out_seq=0 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.461+0000 7fe0d6ffd640 10 client.15540 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1624865873 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ad52c8070 con 0x560ad536f280 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe0d6ffd640 10 client.15540 handle_command_reply: tid=0 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 client.15540 shutdown 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 msgr2=0x560ad535c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8012730 tx=0x7fe0e80046c0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 shutdown_connections 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad536f280 0x560ad5371670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad535c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe0d6ffd640 0 client.15540 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad535caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 --2- 172.21.15.37:0/1624865873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad535c070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.462+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 >> 172.21.15.37:0/1624865873 conn(0x560ad5350410 msgr2=0x560ad5356140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.463+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 shutdown_connections 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.463+0000 7fe100c33740 1 -- 172.21.15.37:0/1624865873 wait complete. 2024-04-26T18:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.463+0000 7fe100c33740 20 client.15540 trim_cache size 0 max 16384 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.477+0000 7fe100c33740 1 Processor -- start 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.477+0000 7fe100c33740 1 -- start start 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad53568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad5376f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81b0e20 con 0x560ad5353f00 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81ae6f0 con 0x560ad5354820 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe100c33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81ac540 con 0x560ad535f0e0 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39682/0 (socket says 172.21.15.37:39682) 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 -- 172.21.15.37:0/2630802708 learned_addr learned my addr 172.21.15.37:0/2630802708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad53568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad5376f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 -- 172.21.15.37:0/2630802708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 msgr2=0x560ad5376f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad5376f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 -- 172.21.15.37:0/2630802708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 msgr2=0x560ad53568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad53568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 -- 172.21.15.37:0/2630802708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad5361840 con 0x560ad5353f00 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad53568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.478+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13657d588ab6dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe0dd7fa640 1 -- 172.21.15.37:0/2630802708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e803c020 con 0x560ad5353f00 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe0dd7fa640 1 -- 172.21.15.37:0/2630802708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e8011e50 con 0x560ad5353f00 2024-04-26T18:26:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe0dd7fa640 1 -- 172.21.15.37:0/2630802708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8034040 con 0x560ad5353f00 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 -- 172.21.15.37:0/2630802708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 msgr2=0x560ad536abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e8005810 comp rx=0 tx=0).stop 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 -- 172.21.15.37:0/2630802708 shutdown_connections 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad5354820 0x560ad53568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad535f0e0 0x560ad5376f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 --2- 172.21.15.37:0/2630802708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad536abd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 -- 172.21.15.37:0/2630802708 >> 172.21.15.37:0/2630802708 conn(0x560ad5350410 msgr2=0x560ad535e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 -- 172.21.15.37:0/2630802708 shutdown_connections 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.479+0000 7fe100c33740 1 -- 172.21.15.37:0/2630802708 wait complete. 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 -- start start 2024-04-26T18:26:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 Processor -- start 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad5363800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad5363d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81acbb0 con 0x560ad5353f00 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81ae020 con 0x560ad535f0e0 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81ad280 con 0x560ad5354820 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe100c33740 10 client.0 _ll_register_callbacks cb 0x7fe0e800b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe0f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39692/0 (socket says 172.21.15.37:39692) 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.480+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1691110378 learned_addr learned my addr 172.21.15.37:0/1691110378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad5363800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad5363d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1691110378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 msgr2=0x560ad5363800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad5363800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 msgr2=0x560ad5363d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad5363d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 -- 172.21.15.37:0/1691110378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad5364570 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f7fff640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad5363d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0f77fe640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b410 tx=0x7fe0e8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7137ef22598b83da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8034070 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fe0e802fa60 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e8011e50 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe0e803c040 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe100c33740 10 client.15546 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ad53d67b0 con 0x560ad5353f00 2024-04-26T18:26:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.481+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e8011e50 con 0x560ad5353f00 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe0e8037070 con 0x560ad5353f00 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 10 client.15546 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 20 client.15546 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad53d6db0 0x560ad53d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 4 client.15546 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ad52c1ee0 con 0x560ad53d6db0 2024-04-26T18:26:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.482+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad53d6db0 0x560ad53d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:07 smithi037 ceph-mon[29171]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:26:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.483+0000 7fe0d6ffd640 10 client.15546 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.483+0000 7fe0f6ffd640 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad53d6db0 0x560ad53d91a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c531e255874981b server_cookie=11f826f72dabfc7b in_seq=0 out_seq=0 2024-04-26T18:26:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe0d6ffd640 1 -- 172.21.15.37:0/1691110378 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ad52c1ee0 con 0x560ad53d6db0 2024-04-26T18:26:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe0d6ffd640 10 client.15546 handle_command_reply: tid=0 2024-04-26T18:26:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 client.15546 shutdown 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 msgr2=0x560ad5367480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b410 tx=0x7fe0e8004250 comp rx=0 tx=0).stop 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 shutdown_connections 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560ad53d6db0 0x560ad53d91a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560ad535f0e0 0x560ad5363d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad5354820 0x560ad5363800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe0d6ffd640 0 client.15546 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 --2- 172.21.15.37:0/1691110378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ad5353f00 0x560ad5367480 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 >> 172.21.15.37:0/1691110378 conn(0x560ad5350410 msgr2=0x560ad535d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 shutdown_connections 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 1 -- 172.21.15.37:0/1691110378 wait complete. 2024-04-26T18:26:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.489+0000 7fe100c33740 20 client.15546 trim_cache size 0 max 16384 2024-04-26T18:26:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:26:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:26:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:26:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 msgr2=0x7fe0e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 0x7fe0e00a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8005fd0 tx=0x7fe0e800b5d0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 msgr2=0x7fe0f8199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec004770 tx=0x7fe0ec0042f0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 shutdown_connections 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fe0e00a3dc0 0x7fe0e00a6280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0f8107820 0x7fe0f819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0f8102220 0x7fe0f8199680 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 --2- 172.21.15.37:0/891766275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0f8101860 0x7fe0f8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 >> 172.21.15.37:0/891766275 conn(0x7fe0f80789b0 msgr2=0x7fe0f806a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 shutdown_connections 2024-04-26T18:26:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:07.490+0000 7fe0fde46640 1 -- 172.21.15.37:0/891766275 wait complete. 2024-04-26T18:26:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:07 smithi120 ceph-mon[32244]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:26:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:07 smithi170 ceph-mon[32697]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:26:08.728 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 94% (11547/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T18:26:09.377 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:09 smithi170 ceph-mon[32697]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:09 smithi037 ceph-mon[29171]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:09 smithi120 ceph-mon[32244]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:09.542 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'. 2024-04-26T18:26:09.542 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T18:26:09.543 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T18:26:09.544 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at bc94a2a924f Merge commit 'refs/pull/57099/head' of github.com:ceph/ceph into quiescer 2024-04-26T18:26:09.548 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T18:26:09.574 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T18:26:09.576 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:26:09.576 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T18:26:10.242 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T18:26:10.296 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T18:26:10.337 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:26:10.339 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:26:10.339 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T18:26:10.378 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:26:10.382 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:26:10.382 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T18:26:10.438 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-26T18:26:10.439 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-26T18:26:10.439 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-26T18:26:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:11 smithi037 ceph-mon[29171]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:11 smithi120 ceph-mon[32244]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:11 smithi170 ceph-mon[32697]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:26:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:13 smithi037 ceph-mon[29171]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:26:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:13 smithi120 ceph-mon[32244]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:26:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:13 smithi170 ceph-mon[32697]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:26:15.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:15 smithi037 ceph-mon[29171]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2024-04-26T18:26:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:15 smithi120 ceph-mon[32244]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2024-04-26T18:26:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:15 smithi170 ceph-mon[32697]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 17 op/s 2024-04-26T18:26:17.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:17 smithi037 ceph-mon[29171]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-04-26T18:26:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:17 smithi120 ceph-mon[32244]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-04-26T18:26:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:17 smithi170 ceph-mon[32697]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.3 KiB/s wr, 20 op/s 2024-04-26T18:26:19.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:19 smithi037 ceph-mon[29171]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:19.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:19 smithi120 ceph-mon[32244]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:19 smithi170 ceph-mon[32697]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- 172.21.15.37:0/970150471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 msgr2=0x7f14d4107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 --2- 172.21.15.37:0/970150471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d4107c40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009930 tx=0x7f14c402a420 comp rx=0 tx=0).stop 2024-04-26T18:26:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- 172.21.15.37:0/970150471 shutdown_connections 2024-04-26T18:26:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 --2- 172.21.15.37:0/970150471 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4101850 0x7f14d4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 --2- 172.21.15.37:0/970150471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4102210 0x7f14d410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 --2- 172.21.15.37:0/970150471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d4107c40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- 172.21.15.37:0/970150471 >> 172.21.15.37:0/970150471 conn(0x7f14d4077bb0 msgr2=0x7f14d4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- 172.21.15.37:0/970150471 shutdown_connections 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- 172.21.15.37:0/970150471 wait complete. 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 Processor -- start 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.590+0000 7f14dab78640 1 -- start start 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4101850 0x7f14d410f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 0x7f14d410f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4112c20 con 0x7f14d4107860 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4112aa0 con 0x7f14d4102210 2024-04-26T18:26:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14dab78640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4112da0 con 0x7f14d4101850 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56190/0 (socket says 172.21.15.37:56190) 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 -- 172.21.15.37:0/2041832343 learned_addr learned my addr 172.21.15.37:0/2041832343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d8914640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4101850 0x7f14d410f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d3fff640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 0x7f14d410f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 -- 172.21.15.37:0/2041832343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4101850 msgr2=0x7f14d410f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4101850 0x7f14d410f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 -- 172.21.15.37:0/2041832343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 msgr2=0x7f14d410f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 0x7f14d410f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 -- 172.21.15.37:0/2041832343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41a7560 con 0x7f14d4107860 2024-04-26T18:26:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d3fff640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 0x7f14d410f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.591+0000 7f14d9115640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fd0 tx=0x7f14cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.592+0000 7f14d1ffb640 1 -- 172.21.15.37:0/2041832343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017070 con 0x7f14d4107860 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.592+0000 7f14d1ffb640 1 -- 172.21.15.37:0/2041832343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc003e20 con 0x7f14d4107860 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.592+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41a7850 con 0x7f14d4107860 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.592+0000 7f14d1ffb640 1 -- 172.21.15.37:0/2041832343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc010460 con 0x7f14d4107860 2024-04-26T18:26:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.592+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a7d50 con 0x7f14d4107860 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.594+0000 7f14d1ffb640 1 -- 172.21.15.37:0/2041832343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14cc003970 con 0x7f14d4107860 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.594+0000 7f14dd969740 1 Processor -- start 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.594+0000 7f14dd969740 1 -- start start 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d1ffb640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 0x7f14bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 0x5584e008d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584dfff3460 con 0x5584e008d820 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584dffad590 con 0x5584e00980e0 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14dd969740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5584dffef420 con 0x5584e008cf00 2024-04-26T18:26:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 0x7f14bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d1ffb640 1 -- 172.21.15.37:0/2041832343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f14cc0d05b0 con 0x7f14d4107860 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 0x5584e008d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56198/0 (socket says 172.21.15.37:56198) 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 -- 172.21.15.37:0/74733694 learned_addr learned my addr 172.21.15.37:0/74733694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d9115640 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 -- 172.21.15.37:0/74733694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 msgr2=0x5584e009a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 -- 172.21.15.37:0/74733694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 msgr2=0x5584e0097ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d8914640 1 -- 172.21.15.37:0/74733694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584dffba0d0 con 0x5584e008cf00 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.595+0000 7f14d3fff640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 0x7f14bc0a6280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14c8014a20 tx=0x7f14c800bbf0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14d8914640 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 0x5584e008d2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14c4031440 tx=0x7f14c4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370e3ccd333b77ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14baffd640 1 -- 172.21.15.37:0/74733694 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4040070 con 0x5584e008cf00 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14baffd640 1 -- 172.21.15.37:0/74733694 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c4031e10 con 0x5584e008cf00 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14baffd640 1 -- 172.21.15.37:0/74733694 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035040 con 0x5584e008cf00 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14dd969740 1 -- 172.21.15.37:0/74733694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 msgr2=0x5584e008d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.596+0000 7f14dd969740 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 0x5584e008d2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14c4031440 tx=0x7f14c4005c00 comp rx=0 tx=0).stop 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- 172.21.15.37:0/74733694 shutdown_connections 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e0097ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- 172.21.15.37:0/74733694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008cf00 0x5584e008d2e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- 172.21.15.37:0/74733694 >> 172.21.15.37:0/74733694 conn(0x5584e0089410 msgr2=0x5584e00897f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- 172.21.15.37:0/74733694 shutdown_connections 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- 172.21.15.37:0/74733694 wait complete. 2024-04-26T18:26:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- start start 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 Processor -- start 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e00980e0 0x5584e0095b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584e009acb0 con 0x5584e008d820 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584dff7ab60 con 0x5584e008cf00 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.597+0000 7f14dd969740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5584e009ae30 con 0x5584e00980e0 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14dd969740 10 client.0 _ll_register_callbacks cb 0x5584dff9f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56202/0 (socket says 172.21.15.37:56202) 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 -- 172.21.15.37:0/334295367 learned_addr learned my addr 172.21.15.37:0/334295367 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d9115640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e00980e0 0x5584e0095b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d8914640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 -- 172.21.15.37:0/334295367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e00980e0 msgr2=0x5584e0095b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e00980e0 0x5584e0095b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 -- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 msgr2=0x5584e00950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 -- 172.21.15.37:0/334295367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584dfd05210 con 0x5584e008d820 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d8914640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14d3fff640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14c8014500 tx=0x7f14c801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da37134742ea18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8005020 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f14c80051c0 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c4031e10 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14c80053c0 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14dd969740 10 client.15558 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.598+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584dfdd1620 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c4031e10 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f14c8020520 con 0x5584e008d820 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 20 client.15558 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e00a81a0 0x5584e00aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 4 client.15558 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584e0001070 con 0x5584e00a81a0 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.599+0000 7f14d8914640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e00a81a0 0x5584e00aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.600+0000 7f14b3fff640 10 client.15558 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.600+0000 7f14d8914640 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e00a81a0 0x5584e00aa5f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2377567253183a server_cookie=ee07b809c16bd58a in_seq=0 out_seq=0 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.600+0000 7f14b3fff640 1 -- 172.21.15.37:0/334295367 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584e0001070 con 0x5584e00a81a0 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.600+0000 7f14b3fff640 10 client.15558 handle_command_reply: tid=0 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.600+0000 7f14dd969740 1 client.15558 shutdown 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 msgr2=0x5584e00955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14c8014500 tx=0x7f14c801b000 comp rx=0 tx=0).stop 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 shutdown_connections 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e00a81a0 0x5584e00aa5f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e00980e0 0x5584e0095b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14b3fff640 0 client.15558 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008d820 0x5584e00955f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 --2- 172.21.15.37:0/334295367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 >> 172.21.15.37:0/334295367 conn(0x5584e0089410 msgr2=0x5584e008f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 shutdown_connections 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 1 -- 172.21.15.37:0/334295367 wait complete. 2024-04-26T18:26:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.601+0000 7f14dd969740 20 client.15558 trim_cache size 0 max 16384 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 Processor -- start 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 -- start start 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e0096ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b2fb0 con 0x5584e00980e0 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b0880 con 0x5584e008cf00 2024-04-26T18:26:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14dd969740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41ae6d0 con 0x5584e008d820 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14d9115640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14d9115640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56212/0 (socket says 172.21.15.37:56212) 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14d9115640 1 -- 172.21.15.37:0/1338977537 learned_addr learned my addr 172.21.15.37:0/1338977537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.616+0000 7f14d8914640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d3fff640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e0096ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 -- 172.21.15.37:0/1338977537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 msgr2=0x5584e0096ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e0096ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 -- 172.21.15.37:0/1338977537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 msgr2=0x5584e00a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 -- 172.21.15.37:0/1338977537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e00aac70 con 0x5584e00980e0 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d3fff640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e0096ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d8914640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14d9115640 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00bf50 tx=0x7f14cc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9210f8ece882bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14ba7fc640 1 -- 172.21.15.37:0/1338977537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc09cbe0 con 0x5584e00980e0 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14ba7fc640 1 -- 172.21.15.37:0/1338977537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc09c4e0 con 0x5584e00980e0 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.617+0000 7f14ba7fc640 1 -- 172.21.15.37:0/1338977537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc098b50 con 0x5584e00980e0 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- 172.21.15.37:0/1338977537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 msgr2=0x5584e00aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00bf50 tx=0x7f14cc0092a0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- 172.21.15.37:0/1338977537 shutdown_connections 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e00980e0 0x5584e00aff00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e0096ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- 172.21.15.37:0/1338977537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e008cf00 0x5584e00a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- 172.21.15.37:0/1338977537 >> 172.21.15.37:0/1338977537 conn(0x5584e0089410 msgr2=0x5584e00968e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- 172.21.15.37:0/1338977537 shutdown_connections 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- 172.21.15.37:0/1338977537 wait complete. 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- start start 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 Processor -- start 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e00a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aed40 con 0x5584e008cf00 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.618+0000 7f14dd969740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b01b0 con 0x5584e00980e0 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14dd969740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41af410 con 0x5584e008d820 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14dd969740 10 client.0 _ll_register_callbacks cb 0x7f14c800b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56222/0 (socket says 172.21.15.37:56222) 2024-04-26T18:26:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 -- 172.21.15.37:0/4273451795 learned_addr learned my addr 172.21.15.37:0/4273451795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d3fff640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e00a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d9115640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 -- 172.21.15.37:0/4273451795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 msgr2=0x5584e00a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e00a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 -- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 msgr2=0x5584e009d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 -- 172.21.15.37:0/4273451795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e009d6f0 con 0x5584e008cf00 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14d8914640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eae1f06f873382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035040 con 0x5584e008cf00 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f14c4004040 con 0x5584e008cf00 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc09c4e0 con 0x5584e008cf00 2024-04-26T18:26:20.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14c405e050 con 0x5584e008cf00 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.619+0000 7f14dd969740 10 client.15564 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584e010f100 con 0x5584e008cf00 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc09c4e0 con 0x5584e008cf00 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f14c4004830 con 0x5584e008cf00 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 20 client.15564 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e010f460 0x5584e0111880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 4 client.15564 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584dfffaee0 con 0x5584e010f460 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14d3fff640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e010f460 0x5584e0111880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14b3fff640 10 client.15564 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:20.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.620+0000 7f14d3fff640 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e010f460 0x5584e0111880 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ae93121bbc1163 server_cookie=1ac97339d43259e5 in_seq=0 out_seq=0 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.626+0000 7f14b3fff640 1 -- 172.21.15.37:0/4273451795 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584dfffaee0 con 0x5584e010f460 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.626+0000 7f14b3fff640 10 client.15564 handle_command_reply: tid=0 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.626+0000 7f14dd969740 1 client.15564 shutdown 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 msgr2=0x5584e00a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c4005020 comp rx=0 tx=0).stop 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 shutdown_connections 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5584e010f460 0x5584e0111880 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584e00980e0 0x5584e009d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5584e008d820 0x5584e00a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 --2- 172.21.15.37:0/4273451795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584e008cf00 0x5584e00a0080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14b3fff640 0 client.15564 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 >> 172.21.15.37:0/4273451795 conn(0x5584e0089410 msgr2=0x5584e00885b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 shutdown_connections 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 1 -- 172.21.15.37:0/4273451795 wait complete. 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dd969740 20 client.15564 trim_cache size 0 max 16384 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:26:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:26:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 msgr2=0x7f14bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 0x7f14bc0a6280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14c8014a20 tx=0x7f14c800bbf0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 msgr2=0x7f14d410fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fd0 tx=0x7f14cc00b260 comp rx=0 tx=0).stop 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 shutdown_connections 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f14bc0a3dc0 0x7f14bc0a6280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14d4102210 0x7f14d410f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d4107860 0x7f14d410fc30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 --2- 172.21.15.37:0/2041832343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14d4101850 0x7f14d410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.627+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 >> 172.21.15.37:0/2041832343 conn(0x7f14d4077bb0 msgr2=0x7f14d40ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.628+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 shutdown_connections 2024-04-26T18:26:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:20.628+0000 7f14dab78640 1 -- 172.21.15.37:0/2041832343 wait complete. 2024-04-26T18:26:21.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:21 smithi037 ceph-mon[29171]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:21.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:21 smithi120 ceph-mon[32244]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:21 smithi170 ceph-mon[32697]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T18:26:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:23 smithi037 ceph-mon[29171]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:26:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:23 smithi120 ceph-mon[32244]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:26:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:23 smithi170 ceph-mon[32697]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:26:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:25 smithi037 ceph-mon[29171]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:26:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:25 smithi120 ceph-mon[32244]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:26:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:25 smithi170 ceph-mon[32697]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T18:26:25.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:26:26.082 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:27.476 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:27 smithi037 ceph-mon[29171]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-26T18:26:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:27 smithi120 ceph-mon[32244]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-26T18:26:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:27 smithi170 ceph-mon[32697]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- 172.21.15.37:0/3824015877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006bad0 msgr2=0x7f2d001102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 --2- 172.21.15.37:0/3824015877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006bad0 0x7f2d001102e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800b0a0 tx=0x7f2cf800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- 172.21.15.37:0/3824015877 shutdown_connections 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 --2- 172.21.15.37:0/3824015877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006bad0 0x7f2d001102e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 --2- 172.21.15.37:0/3824015877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 0x7f2d00109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 --2- 172.21.15.37:0/3824015877 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006b130 0x7f2d0006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- 172.21.15.37:0/3824015877 >> 172.21.15.37:0/3824015877 conn(0x7f2d000792d0 msgr2=0x7f2d000796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- 172.21.15.37:0/3824015877 shutdown_connections 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- 172.21.15.37:0/3824015877 wait complete. 2024-04-26T18:26:27.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 Processor -- start 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.840+0000 7f2d08471700 1 -- start start 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006bad0 0x7f2d0019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 0x7f2d00199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d001135c0 con 0x7f2d0006b130 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00113440 con 0x7f2d0006bad0 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d08471700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00113740 con 0x7f2d001094a0 2024-04-26T18:26:27.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36864/0 (socket says 172.21.15.37:36864) 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 -- 172.21.15.37:0/323888147 learned_addr learned my addr 172.21.15.37:0/323888147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d06a0e700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 0x7f2d00199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d05a0c700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006bad0 0x7f2d0019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 -- 172.21.15.37:0/323888147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 msgr2=0x7f2d00199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 0x7f2d00199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 -- 172.21.15.37:0/323888147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006bad0 msgr2=0x7f2d0019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006bad0 0x7f2d0019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 -- 172.21.15.37:0/323888147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0019a0a0 con 0x7f2d0006b130 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.841+0000 7f2d0620d700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d980 tx=0x7f2cfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.842+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc017070 con 0x7f2d0006b130 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.842+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0019a390 con 0x7f2d0006b130 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.842+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc005020 con 0x7f2d0006b130 2024-04-26T18:26:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.842+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00fa10 con 0x7f2d0006b130 2024-04-26T18:26:27.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.842+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d001a65e0 con 0x7f2d0006b130 2024-04-26T18:26:27.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.843+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0006a110 con 0x7f2d0006b130 2024-04-26T18:26:27.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.844+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2cfc004a30 con 0x7f2d0006b130 2024-04-26T18:26:27.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.844+0000 7f2cf77fe700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 0x7f2cec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:27.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.844+0000 7f2d05a0c700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 0x7f2cec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:27.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.844+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2cfc0d0590 con 0x7f2d0006b130 2024-04-26T18:26:27.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.845+0000 7f2d05a0c700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 0x7f2cec0a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d001022b0 tx=0x7f2cf000a300 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:27.849+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cfc020020 con 0x7f2d0006b130 2024-04-26T18:26:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.120+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d0002e490 con 0x7f2d0006b130 2024-04-26T18:26:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.121+0000 7f2cf77fe700 1 -- 172.21.15.37:0/323888147 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11883 (secure 0 0 0) 0x7f2cfc09cc20 con 0x7f2d0006b130 2024-04-26T18:26:28.122 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:26:28.122 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:24:17.933921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 msgr2=0x7f2cec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 0x7f2cec0a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d001022b0 tx=0x7f2cf000a300 comp rx=0 tx=0).stop 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 msgr2=0x7f2d0019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d980 tx=0x7f2cfc00dc90 comp rx=0 tx=0).stop 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 shutdown_connections 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d001094a0 0x7f2d00199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d0006b130 0x7f2d0019f710 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2cec0a3e30 0x7f2cec0a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 --2- 172.21.15.37:0/323888147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2d0006bad0 0x7f2d0019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 >> 172.21.15.37:0/323888147 conn(0x7f2d000792d0 msgr2=0x7f2d00100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 shutdown_connections 2024-04-26T18:26:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:28.123+0000 7f2d08471700 1 -- 172.21.15.37:0/323888147 wait complete. 2024-04-26T18:26:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-26T18:26:28.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:28 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:28 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:28.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:28 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:28.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:28 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:26:28.434 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:28 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/323888147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:28 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:28 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:28 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:28 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:26:28.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:28 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/323888147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:28 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:28 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:28 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:28 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:26:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:28 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/323888147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:28.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id 33c0af61 --release --await' 2024-04-26T18:26:28.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db --set-id 33c0af61 --release --await 2024-04-26T18:26:29.111 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:29.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:29 smithi037 ceph-mon[29171]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:29 smithi120 ceph-mon[32244]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:29 smithi170 ceph-mon[32697]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 -- 172.21.15.37:0/2292049665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f989406bb50 msgr2=0x7f989406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 --2- 172.21.15.37:0/2292049665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f989406bb50 0x7f989406bfd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9888009990 tx=0x7f9888009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 -- 172.21.15.37:0/2292049665 shutdown_connections 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 --2- 172.21.15.37:0/2292049665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f98941102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 --2- 172.21.15.37:0/2292049665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98941094e0 0x7f98941098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 --2- 172.21.15.37:0/2292049665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f989406bb50 0x7f989406bfd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 -- 172.21.15.37:0/2292049665 >> 172.21.15.37:0/2292049665 conn(0x7f9894078190 msgr2=0x7f98940785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 -- 172.21.15.37:0/2292049665 shutdown_connections 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.841+0000 7f989ac06700 1 -- 172.21.15.37:0/2292049665 wait complete. 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 Processor -- start 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 -- start start 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f989406bb50 0x7f989419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f989419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9894113620 con 0x7f98941094e0 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98941134a0 con 0x7f989406bb50 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f989ac06700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98941137a0 con 0x7f989406c510 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f98991a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f98991a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36876/0 (socket says 172.21.15.37:36876) 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.842+0000 7f98991a3700 1 -- 172.21.15.37:0/2883054796 learned_addr learned my addr 172.21.15.37:0/2883054796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98989a2700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f989406bb50 0x7f989419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f9893fff700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f989419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 -- 172.21.15.37:0/2883054796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 msgr2=0x7f989419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f989419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 -- 172.21.15.37:0/2883054796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f989406bb50 msgr2=0x7f989419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f989406bb50 0x7f989419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 -- 172.21.15.37:0/2883054796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989419a0e0 con 0x7f98941094e0 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f9893fff700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f989419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f98991a3700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f988c007ee0 tx=0x7f988c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f9891ffb700 1 -- 172.21.15.37:0/2883054796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00dbd0 con 0x7f98941094e0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989419a3d0 con 0x7f98941094e0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f9891ffb700 1 -- 172.21.15.37:0/2883054796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c00dd70 con 0x7f98941094e0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f9891ffb700 1 -- 172.21.15.37:0/2883054796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c005990 con 0x7f98941094e0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.843+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a6690 con 0x7f98941094e0 2024-04-26T18:26:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.846+0000 7f9891ffb700 1 -- 172.21.15.37:0/2883054796 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f988c01a020 con 0x7f98941094e0 2024-04-26T18:26:30.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.847+0000 7f9891ffb700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 0x7f987c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.847+0000 7f98989a2700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 0x7f987c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.847+0000 7f9891ffb700 1 -- 172.21.15.37:0/2883054796 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f988c0d0080 con 0x7f98941094e0 2024-04-26T18:26:30.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.847+0000 7f98989a2700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 0x7f987c0a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9884005fd0 tx=0x7f9884005db0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.847+0000 7f98a5498b80 1 Processor -- start 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 -- start start 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1296840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1299170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560cf11da840 con 0x560cf12871a0 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560cf11796e0 con 0x560cf1296d80 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98a5498b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560cf11d6800 con 0x560cf1287ac0 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36878/0 (socket says 172.21.15.37:36878) 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 -- 172.21.15.37:0/683655765 learned_addr learned my addr 172.21.15.37:0/683655765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:30.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f9893fff700 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1296840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98991a3700 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1299170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 -- 172.21.15.37:0/683655765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 msgr2=0x560cf1296840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1296840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 -- 172.21.15.37:0/683655765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 msgr2=0x560cf1299170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1299170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 -- 172.21.15.37:0/683655765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf11279d0 con 0x560cf12871a0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.848+0000 7f98989a2700 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f988400ebb0 tx=0x7f988400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1e793b4ed98fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f987affd700 1 -- 172.21.15.37:0/683655765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884021070 con 0x560cf12871a0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f987affd700 1 -- 172.21.15.37:0/683655765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9884010cd0 con 0x560cf12871a0 2024-04-26T18:26:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f987affd700 1 -- 172.21.15.37:0/683655765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988400f620 con 0x560cf12871a0 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 -- 172.21.15.37:0/683655765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 msgr2=0x560cf1287580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f988400ebb0 tx=0x7f988400af90 comp rx=0 tx=0).stop 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 -- 172.21.15.37:0/683655765 shutdown_connections 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1299170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1287580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 --2- 172.21.15.37:0/683655765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1296840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.849+0000 7f98a5498b80 1 -- 172.21.15.37:0/683655765 >> 172.21.15.37:0/683655765 conn(0x560cf11b4e40 msgr2=0x560cf1283690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- 172.21.15.37:0/683655765 shutdown_connections 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- 172.21.15.37:0/683655765 wait complete. 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- start start 2024-04-26T18:26:30.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 Processor -- start 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1293850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1293d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560cf1299950 con 0x560cf12871a0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560cf1140a50 con 0x560cf1296d80 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560cf1299ad0 con 0x560cf1287ac0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98a5498b80 10 client.0 _ll_register_callbacks cb 0x560cf116e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.850+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36886/0 (socket says 172.21.15.37:36886) 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 -- 172.21.15.37:0/3709103554 learned_addr learned my addr 172.21.15.37:0/3709103554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f9893fff700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1293850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98991a3700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1293d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 -- 172.21.15.37:0/3709103554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 msgr2=0x560cf1293850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1293850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 -- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 msgr2=0x560cf1293d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1293d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 -- 172.21.15.37:0/3709103554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf1294490 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98989a2700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f988400a360 tx=0x7f988401dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0f53599216f098 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f9873fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884034060 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f9884010ad0 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9884010cd0 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9884021050 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98a5498b80 10 client.15582 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.851+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560cf12a4420 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9884010cd0 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f98840374e0 con 0x560cf12871a0 2024-04-26T18:26:30.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 10 client.15582 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 20 client.15582 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf12a6a00 0x560cf12a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 4 client.15582 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cf11e8220 con 0x560cf12a6a00 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.852+0000 7f9893fff700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf12a6a00 0x560cf12a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f9893fff700 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf12a6a00 0x560cf12a8df0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6622c0ab4ea4e370 server_cookie=b13d012bf33e9db6 in_seq=0 out_seq=0 2024-04-26T18:26:30.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f9873fff700 10 client.15582 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f9873fff700 1 -- 172.21.15.37:0/3709103554 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cf11e8220 con 0x560cf12a6a00 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f9873fff700 10 client.15582 handle_command_reply: tid=0 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f98a5498b80 1 client.15582 shutdown 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.853+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 msgr2=0x560cf1296550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f988400a360 tx=0x7f988401dab0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 shutdown_connections 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf12a6a00 0x560cf12a8df0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1293d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1296550 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3709103554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf1293850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 >> 172.21.15.37:0/3709103554 conn(0x560cf11b4e40 msgr2=0x560cf12832c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f9873fff700 0 client.15582 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 shutdown_connections 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 1 -- 172.21.15.37:0/3709103554 wait complete. 2024-04-26T18:26:30.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.854+0000 7f98a5498b80 20 client.15582 trim_cache size 0 max 16384 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 Processor -- start 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 -- start start 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98941ae3e0 con 0x560cf12871a0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98941abc70 con 0x560cf1296d80 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98a5498b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98941a9ac0 con 0x560cf1287ac0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98991a3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f9893fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98989a2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36902/0 (socket says 172.21.15.37:36902) 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98991a3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:51198/0 (socket says 172.21.15.37:51198) 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f98989a2700 1 -- 172.21.15.37:0/3571811370 learned_addr learned my addr 172.21.15.37:0/3571811370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.874+0000 7f9893fff700 1 -- 172.21.15.37:0/3571811370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 msgr2=0x560cf1283780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f9893fff700 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f9893fff700 1 -- 172.21.15.37:0/3571811370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 msgr2=0x560cf1292830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f9893fff700 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f9893fff700 1 -- 172.21.15.37:0/3571811370 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cf1284970 con 0x560cf1287ac0 2024-04-26T18:26:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f98989a2700 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f98991a3700 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f9893fff700 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128f070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9888006010 tx=0x7f988801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cae13c3be8d523 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f987a7fc700 1 -- 172.21.15.37:0/3571811370 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888003790 con 0x560cf1287ac0 2024-04-26T18:26:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f987a7fc700 1 -- 172.21.15.37:0/3571811370 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9888003de0 con 0x560cf1287ac0 2024-04-26T18:26:30.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.875+0000 7f987a7fc700 1 -- 172.21.15.37:0/3571811370 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988800bd80 con 0x560cf1287ac0 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- 172.21.15.37:0/3571811370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 msgr2=0x560cf128f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128f070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9888006010 tx=0x7f988801a040 comp rx=0 tx=0).stop 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- 172.21.15.37:0/3571811370 shutdown_connections 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf1296d80 0x560cf1283780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf12871a0 0x560cf1292830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 --2- 172.21.15.37:0/3571811370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128f070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- 172.21.15.37:0/3571811370 >> 172.21.15.37:0/3571811370 conn(0x560cf11b4e40 msgr2=0x560cf12890a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- 172.21.15.37:0/3571811370 shutdown_connections 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- 172.21.15.37:0/3571811370 wait complete. 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 -- start start 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.876+0000 7f98a5498b80 1 Processor -- start 2024-04-26T18:26:30.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 0x560cf128c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98941aa130 con 0x560cf1296d80 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98941ab5a0 con 0x560cf12871a0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98941aa800 con 0x560cf1287ac0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98a5498b80 10 client.0 _ll_register_callbacks cb 0x560cf116e680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36914/0 (socket says 172.21.15.37:36914) 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 -- 172.21.15.37:0/1679154131 learned_addr learned my addr 172.21.15.37:0/1679154131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98989a2700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 0x560cf128c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f9893fff700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 -- 172.21.15.37:0/1679154131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 msgr2=0x560cf128cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 -- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 msgr2=0x560cf128c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 0x560cf128c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 -- 172.21.15.37:0/1679154131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cf129e310 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98989a2700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 0x560cf128c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.877+0000 7f98991a3700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f988c0989e0 tx=0x7f988c0982b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac0488760b30d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a4c80 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f988c09c270 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9888003de0 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f988c09c470 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 10 client.15588 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560cf129f650 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9888003de0 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f988c0b5290 con 0x560cf1296d80 2024-04-26T18:26:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 10 client.15588 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 20 client.15588 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf13118d0 0x560cf1313cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.878+0000 7f98a5498b80 4 client.15588 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "33c0af61", "release": true, "await": true}] 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.879+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "set_id": "33c0af61", "release": true, "await": true}) -- 0x560cf11e22c0 con 0x560cf13118d0 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.879+0000 7f98989a2700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf13118d0 0x560cf1313cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.879+0000 7f9873fff700 10 client.15588 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:30.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.879+0000 7f98989a2700 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf13118d0 0x560cf1313cf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6dfeb7b7252d92 server_cookie=18be57228d9c3281 in_seq=0 out_seq=0 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f9873fff700 1 -- 172.21.15.37:0/1679154131 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x560cf11e22c0 con 0x560cf13118d0 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f9873fff700 10 client.15588 handle_command_reply: tid=0 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 client.15588 shutdown 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 msgr2=0x560cf129dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f988c0989e0 tx=0x7f988c0982b0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 shutdown_connections 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560cf13118d0 0x560cf1313cf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560cf1296d80 0x560cf129dd50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560cf12871a0 0x560cf128c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 --2- 172.21.15.37:0/1679154131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cf1287ac0 0x560cf128cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f9873fff700 0 client.15588 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 >> 172.21.15.37:0/1679154131 conn(0x560cf11b4e40 msgr2=0x560cf1293100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 shutdown_connections 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 1 -- 172.21.15.37:0/1679154131 wait complete. 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.883+0000 7f98a5498b80 20 client.15588 trim_cache size 0 max 16384 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "33c0af61": { 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-04-26T18:26:30.886 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:26:30.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 msgr2=0x7f987c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 0x7f987c0a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9884005fd0 tx=0x7f9884005db0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 msgr2=0x7f9894199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f988c007ee0 tx=0x7f988c00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 shutdown_connections 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98941094e0 0x7f9894199980 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f987c0a3db0 0x7f987c0a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f989406bb50 0x7f989419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 --2- 172.21.15.37:0/2883054796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f989406c510 0x7f989419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 >> 172.21.15.37:0/2883054796 conn(0x7f9894078190 msgr2=0x7f98941013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 shutdown_connections 2024-04-26T18:26:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:26:30.884+0000 7f989ac06700 1 -- 172.21.15.37:0/2883054796 wait complete. 2024-04-26T18:26:30.889 INFO:tasks.workunit.client.0.smithi037.stderr:+ set -e 2024-04-26T18:26:30.889 INFO:tasks.workunit.client.0.smithi037.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-26T18:26:30.890 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into 'xfstests-dev'... 2024-04-26T18:26:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:31 smithi037 ceph-mon[29171]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:31 smithi120 ceph-mon[32244]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:31 smithi170 ceph-mon[32697]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:31.813 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '33c0af61', seconds elapsed: 42.0 2024-04-26T18:26:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:33 smithi037 ceph-mon[29171]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T18:26:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:33 smithi120 ceph-mon[32244]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T18:26:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:33 smithi170 ceph-mon[32697]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T18:26:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 -- 172.21.15.37:0/3950888107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e6410cf00 msgr2=0x7f6e6410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 --2- 172.21.15.37:0/3950888107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e6410cf00 0x7f6e6410f3a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02a510 comp rx=0 tx=0).stop 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 -- 172.21.15.37:0/3950888107 shutdown_connections 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 --2- 172.21.15.37:0/3950888107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e6410cf00 0x7f6e6410f3a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 --2- 172.21.15.37:0/3950888107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e641044e0 0x7f6e6410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 --2- 172.21.15.37:0/3950888107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e64103bc0 0x7f6e64103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 -- 172.21.15.37:0/3950888107 >> 172.21.15.37:0/3950888107 conn(0x7f6e640789b0 msgr2=0x7f6e64078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 -- 172.21.15.37:0/3950888107 shutdown_connections 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 -- 172.21.15.37:0/3950888107 wait complete. 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.723+0000 7f6e69699640 1 Processor -- start 2024-04-26T18:26:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 -- start start 2024-04-26T18:26:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 0x7f6e641990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 0x7f6e6419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64112bb0 con 0x7f6e641044e0 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64112a30 con 0x7f6e6410cf00 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e69699640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64112d30 con 0x7f6e64103bc0 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36936/0 (socket says 172.21.15.37:36936) 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 -- 172.21.15.37:0/1577544434 learned_addr learned my addr 172.21.15.37:0/1577544434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e637fe640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 0x7f6e6419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 0x7f6e641990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 -- 172.21.15.37:0/1577544434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 msgr2=0x7f6e641990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 0x7f6e641990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 -- 172.21.15.37:0/1577544434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 msgr2=0x7f6e6419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 0x7f6e6419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e627fc640 1 -- 172.21.15.37:0/1577544434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6419e0a0 con 0x7f6e641044e0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 0x7f6e641990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.724+0000 7f6e637fe640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 0x7f6e6419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e627fc640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d770 tx=0x7f6e5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e4bfff640 1 -- 172.21.15.37:0/1577544434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58017070 con 0x7f6e641044e0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e4bfff640 1 -- 172.21.15.37:0/1577544434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e580054d0 con 0x7f6e641044e0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6419e390 con 0x7f6e641044e0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e4bfff640 1 -- 172.21.15.37:0/1577544434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58004c30 con 0x7f6e641044e0 2024-04-26T18:26:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.725+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e641a5c20 con 0x7f6e641044e0 2024-04-26T18:26:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.727+0000 7f6e4bfff640 1 -- 172.21.15.37:0/1577544434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6e58004e10 con 0x7f6e641044e0 2024-04-26T18:26:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.727+0000 7f6e6c3bc740 1 Processor -- start 2024-04-26T18:26:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 -- start start 2024-04-26T18:26:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e1d6451460 con 0x55e1d64eb820 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e1d640b590 con 0x55e1d64f60e0 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e1d644d420 con 0x55e1d64eaf00 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36946/0 (socket says 172.21.15.37:36946) 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 -- 172.21.15.37:0/3040791782 learned_addr learned my addr 172.21.15.37:0/3040791782 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e4bfff640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 0x7f6e4c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e627fc640 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e637fe640 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e637fe640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 0x7f6e4c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 -- 172.21.15.37:0/3040791782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 msgr2=0x55e1d64eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64eb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 -- 172.21.15.37:0/3040791782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 msgr2=0x55e1d64f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e4bfff640 1 -- 172.21.15.37:0/1577544434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f6e580d0270 con 0x7f6e641044e0 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.728+0000 7f6e62ffd640 1 -- 172.21.15.37:0/3040791782 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d64180d0 con 0x55e1d64eb820 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400f390 tx=0x7f6e5400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7509e193634b6f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e497fa640 1 -- 172.21.15.37:0/3040791782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54003b70 con 0x55e1d64eb820 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e637fe640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 0x7f6e4c0a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006010 tx=0x7f6e5c0058d0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e497fa640 1 -- 172.21.15.37:0/3040791782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e54003d10 con 0x55e1d64eb820 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e497fa640 1 -- 172.21.15.37:0/3040791782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e540119c0 con 0x55e1d64eb820 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/3040791782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 msgr2=0x55e1d64f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.729+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400f390 tx=0x7f6e5400f850 comp rx=0 tx=0).stop 2024-04-26T18:26:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/3040791782 shutdown_connections 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d64f5ba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/3040791782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/3040791782 >> 172.21.15.37:0/3040791782 conn(0x55e1d64e7410 msgr2=0x55e1d64e77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/3040791782 shutdown_connections 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/3040791782 wait complete. 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- start start 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 Processor -- start 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eb820 0x55e1d64f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e1d64f8d10 con 0x55e1d64f60e0 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e1d63d8b60 con 0x55e1d64eb820 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e1d64f8e90 con 0x55e1d64eaf00 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e6c3bc740 10 client.0 _ll_register_callbacks cb 0x55e1d63fd2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36960/0 (socket says 172.21.15.37:36960) 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e637fe640 1 -- 172.21.15.37:0/212290305 learned_addr learned my addr 172.21.15.37:0/212290305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.730+0000 7f6e627fc640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eb820 0x55e1d64f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 -- 172.21.15.37:0/212290305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 msgr2=0x55e1d64f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 -- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eb820 msgr2=0x55e1d64f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eb820 0x55e1d64f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 -- 172.21.15.37:0/212290305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d6163210 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e637fe640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fe00cc2bd54836 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c035030 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f6e5c0038b0 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e54003d10 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e5c003ab0 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e6c3bc740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1d622f620 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.731+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e54003d10 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6e5c050430 con 0x55e1d64f60e0 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 20 client.15600 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d65062c0 0x55e1d65086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 4 client.15600 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1d645f070 con 0x55e1d65062c0 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d65062c0 0x55e1d65086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e46ff5640 10 client.15600 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.732+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d65062c0 0x55e1d65086d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39a5b7be772b1db server_cookie=67d3ff63fef9b1a7 in_seq=0 out_seq=0 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e46ff5640 1 -- 172.21.15.37:0/212290305 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1d645f070 con 0x55e1d65062c0 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e46ff5640 10 client.15600 handle_command_reply: tid=0 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 client.15600 shutdown 2024-04-26T18:26:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 msgr2=0x55e1d64f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c009260 comp rx=0 tx=0).stop 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 shutdown_connections 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d65062c0 0x55e1d65086d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eb820 0x55e1d64f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64f60e0 0x55e1d64f3b40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e46ff5640 0 client.15600 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/212290305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eaf00 0x55e1d64f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 >> 172.21.15.37:0/212290305 conn(0x55e1d64e7410 msgr2=0x55e1d64ed140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 shutdown_connections 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/212290305 wait complete. 2024-04-26T18:26:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.733+0000 7f6e6c3bc740 20 client.15600 trim_cache size 0 max 16384 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.748+0000 7f6e6c3bc740 1 Processor -- start 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.748+0000 7f6e6c3bc740 1 -- start start 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.748+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 0x55e1d64eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.748+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64f60e0 0x55e1d650dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641b0e50 con 0x55e1d64eb820 2024-04-26T18:26:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641ae720 con 0x55e1d64eaf00 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641ac570 con 0x55e1d64f60e0 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36966/0 (socket says 172.21.15.37:36966) 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 -- 172.21.15.37:0/4287589501 learned_addr learned my addr 172.21.15.37:0/4287589501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e637fe640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64f60e0 0x55e1d650dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 0x55e1d64eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 -- 172.21.15.37:0/4287589501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64f60e0 msgr2=0x55e1d650dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64f60e0 0x55e1d650dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 -- 172.21.15.37:0/4287589501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 msgr2=0x55e1d64eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 0x55e1d64eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 -- 172.21.15.37:0/4287589501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d64eec50 con 0x55e1d64eb820 2024-04-26T18:26:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 0x55e1d64eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e627fc640 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e58098bd0 tx=0x7f6e5806dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3531c150d64f0e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e48ff9640 1 -- 172.21.15.37:0/4287589501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58020040 con 0x55e1d64eb820 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e48ff9640 1 -- 172.21.15.37:0/4287589501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e5809c200 con 0x55e1d64eb820 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.749+0000 7f6e48ff9640 1 -- 172.21.15.37:0/4287589501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580a49e0 con 0x55e1d64eb820 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/4287589501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 msgr2=0x55e1d6501bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e58098bd0 tx=0x7f6e5806dc40 comp rx=0 tx=0).stop 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/4287589501 shutdown_connections 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64f60e0 0x55e1d650dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eb820 0x55e1d6501bd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/4287589501 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64eaf00 0x55e1d64eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/4287589501 >> 172.21.15.37:0/4287589501 conn(0x55e1d64e7410 msgr2=0x55e1d64f1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/4287589501 shutdown_connections 2024-04-26T18:26:33.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/4287589501 wait complete. 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 -- start start 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.750+0000 7f6e6c3bc740 1 Processor -- start 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eb820 0x55e1d64f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641acbe0 con 0x55e1d64eaf00 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641ae050 con 0x55e1d64f60e0 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641ad2b0 con 0x55e1d64eb820 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 10 client.0 _ll_register_callbacks cb 0x7f6e5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36976/0 (socket says 172.21.15.37:36976) 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 -- 172.21.15.37:0/19471043 learned_addr learned my addr 172.21.15.37:0/19471043 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e637fe640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e627fc640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eb820 0x55e1d64f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 -- 172.21.15.37:0/19471043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eb820 msgr2=0x55e1d64f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eb820 0x55e1d64f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 -- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 msgr2=0x55e1d64f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 -- 172.21.15.37:0/19471043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d656b850 con 0x55e1d64eaf00 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e62ffd640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400f390 tx=0x7f6e54011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219572a42eddb527 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54029840 con 0x55e1d64eaf00 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f6e54004930 con 0x55e1d64eaf00 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5809c200 con 0x55e1d64eaf00 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e54036050 con 0x55e1d64eaf00 2024-04-26T18:26:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.751+0000 7f6e6c3bc740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1d656c9d0 con 0x55e1d64eaf00 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e5809c200 con 0x55e1d64eaf00 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6e54031030 con 0x55e1d64eaf00 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 20 client.15606 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d656d010 0x55e1d656f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 4 client.15606 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1d6458ee0 con 0x55e1d656d010 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e627fc640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d656d010 0x55e1d656f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e627fc640 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d656d010 0x55e1d656f430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edf9c7259b43f1b server_cookie=db6d4b945b4847fe in_seq=0 out_seq=0 2024-04-26T18:26:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.752+0000 7f6e46ff5640 10 client.15606 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e46ff5640 1 -- 172.21.15.37:0/19471043 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1d6458ee0 con 0x55e1d656d010 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e46ff5640 10 client.15606 handle_command_reply: tid=0 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 client.15606 shutdown 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 msgr2=0x55e1d64efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400f390 tx=0x7f6e54011660 comp rx=0 tx=0).stop 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 shutdown_connections 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e1d656d010 0x55e1d656f430 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e1d64f60e0 0x55e1d64f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d64eb820 0x55e1d64f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e46ff5640 0 client.15606 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 --2- 172.21.15.37:0/19471043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1d64eaf00 0x55e1d64efc20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 >> 172.21.15.37:0/19471043 conn(0x55e1d64e7410 msgr2=0x55e1d64ec8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 shutdown_connections 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 1 -- 172.21.15.37:0/19471043 wait complete. 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.969+0000 7f6e6c3bc740 20 client.15606 trim_cache size 0 max 16384 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 msgr2=0x7f6e4c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 0x7f6e4c0a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006010 tx=0x7f6e5c0058d0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 msgr2=0x7f6e641995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d770 tx=0x7f6e5800dc30 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 shutdown_connections 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f6e4c0a3dc0 0x7f6e4c0a6280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e6410cf00 0x7f6e6419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e641044e0 0x7f6e641995f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 --2- 172.21.15.37:0/1577544434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e64103bc0 0x7f6e641990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 >> 172.21.15.37:0/1577544434 conn(0x7f6e640789b0 msgr2=0x7f6e640fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 shutdown_connections 2024-04-26T18:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:33.970+0000 7f6e69699640 1 -- 172.21.15.37:0/1577544434 wait complete. 2024-04-26T18:26:35.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:35 smithi037 ceph-mon[29171]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:35 smithi120 ceph-mon[32244]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:35 smithi170 ceph-mon[32697]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:26:37.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:37 smithi037 ceph-mon[29171]: pgmap v71: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-04-26T18:26:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:37 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:26:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:37 smithi120 ceph-mon[32244]: pgmap v71: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-04-26T18:26:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:37 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:26:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:37 smithi170 ceph-mon[32697]: pgmap v71: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-04-26T18:26:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:37 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:26:39.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:39 smithi037 ceph-mon[29171]: pgmap v72: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:39 smithi120 ceph-mon[32244]: pgmap v72: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:39 smithi170 ceph-mon[32697]: pgmap v72: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:41 smithi037 ceph-mon[29171]: pgmap v73: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:41 smithi120 ceph-mon[32244]: pgmap v73: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:41 smithi170 ceph-mon[32697]: pgmap v73: 97 pgs: 97 active+clean; 16 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-04-26T18:26:43.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:43 smithi037 ceph-mon[29171]: pgmap v74: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-04-26T18:26:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:43 smithi120 ceph-mon[32244]: pgmap v74: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-04-26T18:26:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:43 smithi170 ceph-mon[32697]: pgmap v74: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-04-26T18:26:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:45 smithi037 ceph-mon[29171]: pgmap v75: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-04-26T18:26:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:45 smithi120 ceph-mon[32244]: pgmap v75: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-04-26T18:26:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:45 smithi170 ceph-mon[32697]: pgmap v75: 97 pgs: 97 active+clean; 18 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-04-26T18:26:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 -- 172.21.15.37:0/3721604289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88102210 msgr2=0x7fce8810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 --2- 172.21.15.37:0/3721604289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88102210 0x7fce8810e720 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fce80009cd0 tx=0x7fce8002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 -- 172.21.15.37:0/3721604289 shutdown_connections 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 --2- 172.21.15.37:0/3721604289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88102210 0x7fce8810e720 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 --2- 172.21.15.37:0/3721604289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88101850 0x7fce88101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 --2- 172.21.15.37:0/3721604289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88107860 0x7fce88107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 -- 172.21.15.37:0/3721604289 >> 172.21.15.37:0/3721604289 conn(0x7fce88077bb0 msgr2=0x7fce88077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.069+0000 7fce8e42a640 1 -- 172.21.15.37:0/3721604289 shutdown_connections 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.070+0000 7fce8e42a640 1 -- 172.21.15.37:0/3721604289 wait complete. 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.070+0000 7fce8e42a640 1 Processor -- start 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.070+0000 7fce8e42a640 1 -- start start 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 0x7fce88103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88107860 0x7fce88103ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce88111b20 con 0x7fce88101850 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce881119a0 con 0x7fce88102210 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8e42a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce88111ca0 con 0x7fce88107860 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41452/0 (socket says 172.21.15.37:41452) 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 -- 172.21.15.37:0/1155738595 learned_addr learned my addr 172.21.15.37:0/1155738595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce877fe640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 0x7fce88103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88107860 0x7fce88103ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 -- 172.21.15.37:0/1155738595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88107860 msgr2=0x7fce88103ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88107860 0x7fce88103ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 -- 172.21.15.37:0/1155738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 msgr2=0x7fce88103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 0x7fce88103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 -- 172.21.15.37:0/1155738595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce881a7570 con 0x7fce88101850 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce877fe640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 0x7fce88103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.071+0000 7fce87fff640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d950 tx=0x7fce7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.072+0000 7fce857fa640 1 -- 172.21.15.37:0/1155738595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78017030 con 0x7fce88101850 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.072+0000 7fce857fa640 1 -- 172.21.15.37:0/1155738595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78004020 con 0x7fce88101850 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.072+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce881a7860 con 0x7fce88101850 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.072+0000 7fce857fa640 1 -- 172.21.15.37:0/1155738595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780047a0 con 0x7fce88101850 2024-04-26T18:26:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.072+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a7db0 con 0x7fce88101850 2024-04-26T18:26:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.074+0000 7fce857fa640 1 -- 172.21.15.37:0/1155738595 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce78009a30 con 0x7fce88101850 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.074+0000 7fce91222740 1 Processor -- start 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 -- start start 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772dd0e0 0x562f772df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562f77238460 con 0x562f772d1f00 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f771f2590 con 0x562f772dd0e0 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce91222740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562f77234420 con 0x562f772d2820 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce857fa640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 0x7fce700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41456/0 (socket says 172.21.15.37:41456) 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2137007249 learned_addr learned my addr 172.21.15.37:0/2137007249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 0x7fce700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce857fa640 1 -- 172.21.15.37:0/1155738595 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fce780d0300 con 0x7fce88101850 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce87fff640 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce877fe640 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772dd0e0 0x562f772df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2137007249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 msgr2=0x562f772dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2137007249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772dd0e0 msgr2=0x562f772df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772dd0e0 0x562f772df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.075+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2137007249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f771ff0d0 con 0x562f772d1f00 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce80033040 tx=0x7fce80031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68571b28bc5041a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 0x7fce700a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce80035000 tx=0x7fce80048000 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce6e7fc640 1 -- 172.21.15.37:0/2137007249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8004e070 con 0x562f772d1f00 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce6e7fc640 1 -- 172.21.15.37:0/2137007249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80036040 con 0x562f772d1f00 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce6e7fc640 1 -- 172.21.15.37:0/2137007249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004850 con 0x562f772d1f00 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce91222740 1 -- 172.21.15.37:0/2137007249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 msgr2=0x562f772d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.076+0000 7fce91222740 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce80033040 tx=0x7fce80031a60 comp rx=0 tx=0).stop 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 -- 172.21.15.37:0/2137007249 shutdown_connections 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772dd0e0 0x562f772df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 --2- 172.21.15.37:0/2137007249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772d1f00 0x562f772d22e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 -- 172.21.15.37:0/2137007249 >> 172.21.15.37:0/2137007249 conn(0x562f772ce410 msgr2=0x562f772ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 -- 172.21.15.37:0/2137007249 shutdown_connections 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 -- 172.21.15.37:0/2137007249 wait complete. 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 -- start start 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 Processor -- start 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d1f00 0x562f772da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.077+0000 7fce91222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d2820 0x562f772da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562f772dfcb0 con 0x562f772dd0e0 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f771bfb60 con 0x562f772d2820 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562f772dfe30 con 0x562f772d1f00 2024-04-26T18:26:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 10 client.0 _ll_register_callbacks cb 0x562f771e42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41470/0 (socket says 172.21.15.37:41470) 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 -- 172.21.15.37:0/3543646455 learned_addr learned my addr 172.21.15.37:0/3543646455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce87fff640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d1f00 0x562f772da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce877fe640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d2820 0x562f772da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 -- 172.21.15.37:0/3543646455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d1f00 msgr2=0x562f772da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d1f00 0x562f772da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 -- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d2820 msgr2=0x562f772da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d2820 0x562f772da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 -- 172.21.15.37:0/3543646455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f76f4a210 con 0x562f772dd0e0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce8002aff0 tx=0x7fce800048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e27bd3b8d7aca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8004e040 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce80036040 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.078+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fce80004250 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce80030680 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f77016620 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80036040 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fce80030070 con 0x562f772dd0e0 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 20 client.15618 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f772ed160 0x562f772ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 4 client.15618 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f77246070 con 0x562f772ed160 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.079+0000 7fce87fff640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f772ed160 0x562f772ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.080+0000 7fce87fff640 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f772ed160 0x562f772ef550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c7b401c8d91fa6 server_cookie=673ec8f7f8feb1d6 in_seq=0 out_seq=0 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.080+0000 7fce677fe640 10 client.15618 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce677fe640 1 -- 172.21.15.37:0/3543646455 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f77246070 con 0x562f772ed160 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce677fe640 10 client.15618 handle_command_reply: tid=0 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 client.15618 shutdown 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 msgr2=0x562f772dabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce8002aff0 tx=0x7fce800048a0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 shutdown_connections 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f772ed160 0x562f772ef550 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d2820 0x562f772da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772dabf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce677fe640 0 client.15618 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 --2- 172.21.15.37:0/3543646455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d1f00 0x562f772da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 >> 172.21.15.37:0/3543646455 conn(0x562f772ce410 msgr2=0x562f772d4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 shutdown_connections 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 1 -- 172.21.15.37:0/3543646455 wait complete. 2024-04-26T18:26:47.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.081+0000 7fce91222740 20 client.15618 trim_cache size 0 max 16384 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 Processor -- start 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 -- start start 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b3080 con 0x562f772dd0e0 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b0950 con 0x562f772d1f00 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.096+0000 7fce91222740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce881ae7a0 con 0x562f772d2820 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41478/0 (socket says 172.21.15.37:41478) 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2761400177 learned_addr learned my addr 172.21.15.37:0/2761400177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce877fe640 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce87fff640 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2761400177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 msgr2=0x562f772f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2761400177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 msgr2=0x562f772cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 -- 172.21.15.37:0/2761400177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f772df840 con 0x562f772dd0e0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce80030360 tx=0x7fce80031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66db53bf7742e59b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce6dffb640 1 -- 172.21.15.37:0/2761400177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8004e040 con 0x562f772dd0e0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce6dffb640 1 -- 172.21.15.37:0/2761400177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80036040 con 0x562f772dd0e0 2024-04-26T18:26:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.097+0000 7fce6dffb640 1 -- 172.21.15.37:0/2761400177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80004410 con 0x562f772dd0e0 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- 172.21.15.37:0/2761400177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 msgr2=0x562f772f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce80030360 tx=0x7fce80031e40 comp rx=0 tx=0).stop 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- 172.21.15.37:0/2761400177 shutdown_connections 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f772f5200 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 --2- 172.21.15.37:0/2761400177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- 172.21.15.37:0/2761400177 >> 172.21.15.37:0/2761400177 conn(0x562f772ce410 msgr2=0x562f772da9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- 172.21.15.37:0/2761400177 shutdown_connections 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- 172.21.15.37:0/2761400177 wait complete. 2024-04-26T18:26:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 -- start start 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.098+0000 7fce91222740 1 Processor -- start 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce881aee10 con 0x562f772dd0e0 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b0280 con 0x562f772d1f00 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce881af4e0 con 0x562f772d2820 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 10 client.0 _ll_register_callbacks cb 0x7fce7c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41488/0 (socket says 172.21.15.37:41488) 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 -- 172.21.15.37:0/1971121234 learned_addr learned my addr 172.21.15.37:0/1971121234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce877fe640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce87fff640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 -- 172.21.15.37:0/1971121234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 msgr2=0x562f772d79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772d79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 -- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 msgr2=0x562f772d7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772d7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 -- 172.21.15.37:0/1971121234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f77352a70 con 0x562f772dd0e0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce877fe640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce87fff640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce8c9c7640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce8004a600 tx=0x7fce80031dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ec55293de74c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8004e040 con 0x562f772dd0e0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fce8002e040 con 0x562f772dd0e0 2024-04-26T18:26:47.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.099+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce80036040 con 0x562f772dd0e0 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce80003e30 con 0x562f772dd0e0 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 10 client.15624 fetch_fsmap learned FSMap version 27 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562f77353c80 con 0x562f772dd0e0 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80036040 con 0x562f772dd0e0 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fce80030040 con 0x562f772dd0e0 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 20 client.15624 populate_metadata read hostname 'smithi037' 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f77354210 0x562f77356630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 4 client.15624 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f7723fee0 con 0x562f77354210 2024-04-26T18:26:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.100+0000 7fce87fff640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f77354210 0x562f77356630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.101+0000 7fce677fe640 10 client.15624 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.101+0000 7fce87fff640 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f77354210 0x562f77356630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d460d4b1f0d7fbca server_cookie=9c55683ce509c489 in_seq=0 out_seq=0 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce677fe640 1 -- 172.21.15.37:0/1971121234 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f7723fee0 con 0x562f77354210 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce677fe640 10 client.15624 handle_command_reply: tid=0 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 client.15624 shutdown 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 msgr2=0x562f77352530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce8004a600 tx=0x7fce80031dd0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 shutdown_connections 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x562f77354210 0x562f77356630 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562f772dd0e0 0x562f77352530 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562f772d2820 0x562f772d79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce677fe640 0 client.15624 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 --2- 172.21.15.37:0/1971121234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f772d1f00 0x562f772d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 >> 172.21.15.37:0/1971121234 conn(0x562f772ce410 msgr2=0x562f772d9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 shutdown_connections 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 1 -- 172.21.15.37:0/1971121234 wait complete. 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.113+0000 7fce91222740 20 client.15624 trim_cache size 0 max 16384 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:26:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:26:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:26:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 msgr2=0x7fce700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 0x7fce700a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce80035000 tx=0x7fce80048000 comp rx=0 tx=0).stop 2024-04-26T18:26:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 msgr2=0x7fce88103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce7800d950 tx=0x7fce7800de10 comp rx=0 tx=0).stop 2024-04-26T18:26:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 shutdown_connections 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fce700a3dc0 0x7fce700a6280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce88102210 0x7fce88103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce88107860 0x7fce88103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 --2- 172.21.15.37:0/1155738595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce88101850 0x7fce88103420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 >> 172.21.15.37:0/1155738595 conn(0x7fce88077bb0 msgr2=0x7fce8806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 shutdown_connections 2024-04-26T18:26:47.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:26:47.114+0000 7fce8e42a640 1 -- 172.21.15.37:0/1155738595 wait complete. 2024-04-26T18:26:47.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:47 smithi037 ceph-mon[29171]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-26T18:26:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:47 smithi120 ceph-mon[32244]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-26T18:26:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:47 smithi170 ceph-mon[32697]: pgmap v76: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.8 MiB/s wr, 116 op/s 2024-04-26T18:26:49.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:49 smithi037 ceph-mon[29171]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:49 smithi120 ceph-mon[32244]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:49 smithi170 ceph-mon[32697]: pgmap v77: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:51.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:51 smithi037 ceph-mon[29171]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:51 smithi120 ceph-mon[32244]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:51 smithi170 ceph-mon[32697]: pgmap v78: 97 pgs: 97 active+clean; 21 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 613 KiB/s wr, 31 op/s 2024-04-26T18:26:53.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:53 smithi037 ceph-mon[29171]: pgmap v79: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 670 KiB/s wr, 40 op/s 2024-04-26T18:26:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:53 smithi120 ceph-mon[32244]: pgmap v79: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 670 KiB/s wr, 40 op/s 2024-04-26T18:26:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:53 smithi170 ceph-mon[32697]: pgmap v79: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 670 KiB/s wr, 40 op/s 2024-04-26T18:26:54.734 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 16% (712/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 45% (1957/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 56% (2406/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 88% (3781/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 92% (3954/4290) Updating files: 92% (3955/4290) Updating files: 93% (3990/4290) Updating files: 93% (3994/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-26T18:26:54.742 INFO:tasks.workunit.client.0.smithi037.stderr:+ cd xfstests-dev 2024-04-26T18:26:54.774 INFO:tasks.workunit.client.0.smithi037.stderr:+ make -j4 2024-04-26T18:26:54.785 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize -cfi 2024-04-26T18:26:55.154 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-26T18:26:55.154 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file './config.guess' 2024-04-26T18:26:55.165 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file './config.sub' 2024-04-26T18:26:55.178 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file './install-sh' 2024-04-26T18:26:55.190 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file './ltmain.sh' 2024-04-26T18:26:55.226 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-26T18:26:55.226 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-26T18:26:55.233 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-26T18:26:55.239 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-26T18:26:55.244 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-26T18:26:55.250 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-26T18:26:55.256 INFO:tasks.workunit.client.0.smithi037.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-26T18:26:55.258 INFO:tasks.workunit.client.0.smithi037.stdout:cp include/install-sh . 2024-04-26T18:26:55.266 INFO:tasks.workunit.client.0.smithi037.stdout:aclocal -I m4 2024-04-26T18:26:55.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:55 smithi037 ceph-mon[29171]: pgmap v80: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 414 KiB/s wr, 29 op/s 2024-04-26T18:26:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:55 smithi120 ceph-mon[32244]: pgmap v80: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 414 KiB/s wr, 29 op/s 2024-04-26T18:26:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:55 smithi170 ceph-mon[32697]: pgmap v80: 97 pgs: 97 active+clean; 22 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 414 KiB/s wr, 29 op/s 2024-04-26T18:26:56.357 INFO:tasks.workunit.client.0.smithi037.stdout:autoheader 2024-04-26T18:26:56.695 INFO:tasks.workunit.client.0.smithi037.stdout:autoconf 2024-04-26T18:26:56.889 INFO:tasks.workunit.client.0.smithi037.stdout:./configure \ 2024-04-26T18:26:56.889 INFO:tasks.workunit.client.0.smithi037.stdout: --libexecdir=/usr/lib \ 2024-04-26T18:26:56.889 INFO:tasks.workunit.client.0.smithi037.stdout: --exec_prefix=/var/lib 2024-04-26T18:26:57.062 INFO:tasks.workunit.client.0.smithi037.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-26T18:26:57.062 INFO:tasks.workunit.client.0.smithi037.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-26T18:26:57.063 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to print strings... printf 2024-04-26T18:26:57.063 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gcc... gcc 2024-04-26T18:26:57.186 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether the C compiler works... yes 2024-04-26T18:26:57.186 INFO:tasks.workunit.client.0.smithi037.stdout:checking for C compiler default output file name... a.out 2024-04-26T18:26:57.242 INFO:tasks.workunit.client.0.smithi037.stdout:checking for suffix of executables... 2024-04-26T18:26:57.292 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether we are cross compiling... no 2024-04-26T18:26:57.340 INFO:tasks.workunit.client.0.smithi037.stdout:checking for suffix of object files... o 2024-04-26T18:26:57.378 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T18:26:57.412 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether gcc accepts -g... yes 2024-04-26T18:26:57.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:57 smithi037 ceph-mon[29171]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-04-26T18:26:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:57 smithi120 ceph-mon[32244]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-04-26T18:26:57.506 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T18:26:57.510 INFO:tasks.workunit.client.0.smithi037.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-26T18:26:57.514 INFO:tasks.workunit.client.0.smithi037.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T18:26:57.516 INFO:tasks.workunit.client.0.smithi037.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T18:26:57.517 INFO:tasks.workunit.client.0.smithi037.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-26T18:26:57.528 INFO:tasks.workunit.client.0.smithi037.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-26T18:26:57.529 INFO:tasks.workunit.client.0.smithi037.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-26T18:26:57.535 INFO:tasks.workunit.client.0.smithi037.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-26T18:26:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:57 smithi170 ceph-mon[32697]: pgmap v81: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 383 op/s 2024-04-26T18:26:58.416 INFO:tasks.workunit.client.0.smithi037.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-26T18:26:58.416 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether ln -s works... yes 2024-04-26T18:26:58.438 INFO:tasks.workunit.client.0.smithi037.stdout:checking the maximum length of command line arguments... 1572864 2024-04-26T18:26:58.442 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-26T18:26:58.442 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-26T18:26:58.442 INFO:tasks.workunit.client.0.smithi037.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-26T18:26:58.443 INFO:tasks.workunit.client.0.smithi037.stdout:checking for objdump... objdump 2024-04-26T18:26:58.444 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to recognize dependent libraries... pass_all 2024-04-26T18:26:58.444 INFO:tasks.workunit.client.0.smithi037.stdout:checking for dlltool... no 2024-04-26T18:26:58.444 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-26T18:26:58.444 INFO:tasks.workunit.client.0.smithi037.stdout:checking for ar... ar 2024-04-26T18:26:58.551 INFO:tasks.workunit.client.0.smithi037.stdout:checking for archiver @FILE support... @ 2024-04-26T18:26:58.551 INFO:tasks.workunit.client.0.smithi037.stdout:checking for strip... strip 2024-04-26T18:26:58.552 INFO:tasks.workunit.client.0.smithi037.stdout:checking for ranlib... ranlib 2024-04-26T18:26:58.552 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gawk... gawk 2024-04-26T18:26:58.694 INFO:tasks.workunit.client.0.smithi037.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-26T18:26:58.703 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sysroot... no 2024-04-26T18:26:58.724 INFO:tasks.workunit.client.0.smithi037.stdout:checking for a working dd... /usr/bin/dd 2024-04-26T18:26:58.736 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-26T18:26:58.775 INFO:tasks.workunit.client.0.smithi037.stdout:checking for mt... no 2024-04-26T18:26:58.792 INFO:tasks.workunit.client.0.smithi037.stdout:checking if : is a manifest tool... no 2024-04-26T18:26:58.891 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T18:26:59.135 INFO:tasks.workunit.client.0.smithi037.stdout:checking for ANSI C header files... yes 2024-04-26T18:26:59.181 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/types.h... yes 2024-04-26T18:26:59.233 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/stat.h... yes 2024-04-26T18:26:59.283 INFO:tasks.workunit.client.0.smithi037.stdout:checking for stdlib.h... yes 2024-04-26T18:26:59.335 INFO:tasks.workunit.client.0.smithi037.stdout:checking for string.h... yes 2024-04-26T18:26:59.388 INFO:tasks.workunit.client.0.smithi037.stdout:checking for memory.h... yes 2024-04-26T18:26:59.441 INFO:tasks.workunit.client.0.smithi037.stdout:checking for strings.h... yes 2024-04-26T18:26:59.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:26:59 smithi037 ceph-mon[29171]: pgmap v82: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:26:59.497 INFO:tasks.workunit.client.0.smithi037.stdout:checking for inttypes.h... yes 2024-04-26T18:26:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:26:59 smithi120 ceph-mon[32244]: pgmap v82: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:26:59.551 INFO:tasks.workunit.client.0.smithi037.stdout:checking for stdint.h... yes 2024-04-26T18:26:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:26:59 smithi170 ceph-mon[32697]: pgmap v82: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:26:59.604 INFO:tasks.workunit.client.0.smithi037.stdout:checking for unistd.h... yes 2024-04-26T18:26:59.668 INFO:tasks.workunit.client.0.smithi037.stdout:checking for dlfcn.h... yes 2024-04-26T18:26:59.676 INFO:tasks.workunit.client.0.smithi037.stdout:checking for objdir... .libs 2024-04-26T18:26:59.835 INFO:tasks.workunit.client.0.smithi037.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-26T18:26:59.835 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-26T18:26:59.884 INFO:tasks.workunit.client.0.smithi037.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-26T18:26:59.947 INFO:tasks.workunit.client.0.smithi037.stdout:checking if gcc static flag -static works... no 2024-04-26T18:27:00.017 INFO:tasks.workunit.client.0.smithi037.stdout:checking if gcc supports -c -o file.o... yes 2024-04-26T18:27:00.017 INFO:tasks.workunit.client.0.smithi037.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-26T18:27:00.023 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-26T18:27:00.093 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether -lc should be explicitly linked in... no 2024-04-26T18:27:00.184 INFO:tasks.workunit.client.0.smithi037.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-26T18:27:00.184 INFO:tasks.workunit.client.0.smithi037.stdout:checking how to hardcode library paths into programs... immediate 2024-04-26T18:27:00.185 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether stripping libraries is possible... yes 2024-04-26T18:27:00.185 INFO:tasks.workunit.client.0.smithi037.stdout:checking if libtool supports shared libraries... yes 2024-04-26T18:27:00.185 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether to build shared libraries... yes 2024-04-26T18:27:00.186 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether to build static libraries... yes 2024-04-26T18:27:00.192 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gcc... (cached) gcc 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- 172.21.15.37:0/1213211671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c103300 msgr2=0x7f943c10f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 --2- 172.21.15.37:0/1213211671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c103300 0x7f943c10f810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9434009cd0 tx=0x7f943402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- 172.21.15.37:0/1213211671 shutdown_connections 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 --2- 172.21.15.37:0/1213211671 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c102940 0x7f943c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 --2- 172.21.15.37:0/1213211671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c103300 0x7f943c10f810 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 --2- 172.21.15.37:0/1213211671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c108950 0x7f943c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- 172.21.15.37:0/1213211671 >> 172.21.15.37:0/1213211671 conn(0x7f943c079dc0 msgr2=0x7f943c07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- 172.21.15.37:0/1213211671 shutdown_connections 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- 172.21.15.37:0/1213211671 wait complete. 2024-04-26T18:27:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 Processor -- start 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.207+0000 7f9443b71640 1 -- start start 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c103300 0x7f943c075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f943c112c60 con 0x7f943c108950 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f943c112ae0 con 0x7f943c103300 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f943c112de0 con 0x7f943c102940 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60818/0 (socket says 172.21.15.37:60818) 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944210e640 1 -- 172.21.15.37:0/2927799708 learned_addr learned my addr 172.21.15.37:0/2927799708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:38436/0 (socket says 172.21.15.37:38436) 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944110c640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c103300 0x7f943c075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 -- 172.21.15.37:0/2927799708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c103300 msgr2=0x7f943c075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c103300 0x7f943c075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 -- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 msgr2=0x7f943c0756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 -- 172.21.15.37:0/2927799708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c1a7520 con 0x7f943c102940 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944210e640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f944190d640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f943800e9f0 tx=0x7f943800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9432ffd640 1 -- 172.21.15.37:0/2927799708 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943800ccc0 con 0x7f943c102940 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.208+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c1a7810 con 0x7f943c102940 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.209+0000 7f9432ffd640 1 -- 172.21.15.37:0/2927799708 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9438005020 con 0x7f943c102940 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.209+0000 7f9432ffd640 1 -- 172.21.15.37:0/2927799708 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438004a00 con 0x7f943c102940 2024-04-26T18:27:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.209+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c1a7d60 con 0x7f943c102940 2024-04-26T18:27:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.211+0000 7f9432ffd640 1 -- 172.21.15.37:0/2927799708 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9438004ba0 con 0x7f943c102940 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.211+0000 7f9446966740 1 Processor -- start 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 -- start start 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9432ffd640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 0x7f94280a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df6307fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df630824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df62fdb460 con 0x55df630800e0 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df62f95590 con 0x55df63075820 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9446966740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55df62fd7420 con 0x55df63074f00 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f9432ffd640 1 -- 172.21.15.37:0/2927799708 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f94380d0790 con 0x7f943c102940 2024-04-26T18:27:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944210e640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 0x7f94280a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944110c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df6307fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:38446/0 (socket says 172.21.15.37:38446) 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df630824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 -- 172.21.15.37:0/4278119275 learned_addr learned my addr 172.21.15.37:0/4278119275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 -- 172.21.15.37:0/4278119275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 msgr2=0x55df6307fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df6307fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 -- 172.21.15.37:0/4278119275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 msgr2=0x55df630824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df630824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 -- 172.21.15.37:0/4278119275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df62fa20d0 con 0x55df63074f00 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944190d640 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94380a4e60 tx=0x7f943809c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90b5780430e11d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.212+0000 7f944210e640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 0x7f94280a62b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9434009a60 tx=0x7f94340099f0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9427fff640 1 -- 172.21.15.37:0/4278119275 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943804d070 con 0x55df63074f00 2024-04-26T18:27:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9427fff640 1 -- 172.21.15.37:0/4278119275 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943809c720 con 0x55df63074f00 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9427fff640 1 -- 172.21.15.37:0/4278119275 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943801c070 con 0x55df63074f00 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 -- 172.21.15.37:0/4278119275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 msgr2=0x55df630752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94380a4e60 tx=0x7f943809c140 comp rx=0 tx=0).stop 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 -- 172.21.15.37:0/4278119275 shutdown_connections 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df6307fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df630824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 --2- 172.21.15.37:0/4278119275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df630752e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.213+0000 7f9446966740 1 -- 172.21.15.37:0/4278119275 >> 172.21.15.37:0/4278119275 conn(0x55df63071410 msgr2=0x55df630717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- 172.21.15.37:0/4278119275 shutdown_connections 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- 172.21.15.37:0/4278119275 wait complete. 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- start start 2024-04-26T18:27:00.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 Processor -- start 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 0x55df6307db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df63082cb0 con 0x55df63074f00 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55df62f62b60 con 0x55df630800e0 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55df63082e30 con 0x55df63075820 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.214+0000 7f9446966740 10 client.0 _ll_register_callbacks cb 0x55df62f872b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944190d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 0x55df6307db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944190d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60844/0 (socket says 172.21.15.37:60844) 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944190d640 1 -- 172.21.15.37:0/3953683024 learned_addr learned my addr 172.21.15.37:0/3953683024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:00.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944110c640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 -- 172.21.15.37:0/3953683024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 msgr2=0x55df6307d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 -- 172.21.15.37:0/3953683024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 msgr2=0x55df6307d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 -- 172.21.15.37:0/3953683024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df62ced210 con 0x55df630800e0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944190d640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f944210e640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 0x55df6307db60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9434033040 tx=0x7f9434005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145f277f94b7ffe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f94257fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434030d80 con 0x55df630800e0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943809c720 con 0x55df630800e0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f9434049850 con 0x55df630800e0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9434049a50 con 0x55df630800e0 2024-04-26T18:27:00.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f9446966740 10 client.25262 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.215+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df62db9620 con 0x55df630800e0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943809c720 con 0x55df630800e0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9434030070 con 0x55df630800e0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 10 client.25262 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 20 client.25262 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630902e0 0x55df630926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 4 client.25262 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df62fe9070 con 0x55df630902e0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f944190d640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630902e0 0x55df630926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f94257fa640 10 client.25262 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.216+0000 7f944190d640 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630902e0 0x55df630926d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b37b787a1076bf3 server_cookie=12ac9813b7b68697 in_seq=0 out_seq=0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f94257fa640 1 -- 172.21.15.37:0/3953683024 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df62fe9070 con 0x55df630902e0 2024-04-26T18:27:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f94257fa640 10 client.25262 handle_command_reply: tid=0 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 client.25262 shutdown 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 msgr2=0x55df6307db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 0x55df6307db60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9434033040 tx=0x7f9434005d80 comp rx=0 tx=0).stop 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 shutdown_connections 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630902e0 0x55df630926d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df630800e0 0x55df6307db60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 --2- 172.21.15.37:0/3953683024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df63074f00 0x55df6307d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f94257fa640 0 client.25262 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 >> 172.21.15.37:0/3953683024 conn(0x55df63071410 msgr2=0x55df63077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 shutdown_connections 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.217+0000 7f9446966740 1 -- 172.21.15.37:0/3953683024 wait complete. 2024-04-26T18:27:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.218+0000 7f9446966740 20 client.25262 trim_cache size 0 max 16384 2024-04-26T18:27:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 Processor -- start 2024-04-26T18:27:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 -- start start 2024-04-26T18:27:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df6308bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df630778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1b30a0 con 0x55df630800e0 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1b0970 con 0x55df63075820 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f9446966740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1ae7c0 con 0x55df63074f00 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f944110c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df630778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60858/0 (socket says 172.21.15.37:60858) 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f944210e640 1 -- 172.21.15.37:0/2542904109 learned_addr learned my addr 172.21.15.37:0/2542904109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.233+0000 7f944190d640 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df6308bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 -- 172.21.15.37:0/2542904109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 msgr2=0x55df6308bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df6308bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 -- 172.21.15.37:0/2542904109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 msgr2=0x55df63097fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 -- 172.21.15.37:0/2542904109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6306cd60 con 0x55df63075820 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944210e640 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f944110c640 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df630778c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55df6307c980 tx=0x7f942c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d071c369f7e520f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f94277fe640 1 -- 172.21.15.37:0/2542904109 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c00f440 con 0x55df63075820 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f94277fe640 1 -- 172.21.15.37:0/2542904109 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f942c021070 con 0x55df63075820 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.234+0000 7f94277fe640 1 -- 172.21.15.37:0/2542904109 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c01c930 con 0x55df63075820 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- 172.21.15.37:0/2542904109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 msgr2=0x55df630778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df630778c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55df6307c980 tx=0x7f942c005f90 comp rx=0 tx=0).stop 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- 172.21.15.37:0/2542904109 shutdown_connections 2024-04-26T18:27:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63075820 0x55df630778c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df63097fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 --2- 172.21.15.37:0/2542904109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63074f00 0x55df6308bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- 172.21.15.37:0/2542904109 >> 172.21.15.37:0/2542904109 conn(0x55df63071410 msgr2=0x55df6307f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- 172.21.15.37:0/2542904109 shutdown_connections 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- 172.21.15.37:0/2542904109 wait complete. 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 -- start start 2024-04-26T18:27:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.235+0000 7f9446966740 1 Processor -- start 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df6307aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1aee30 con 0x55df630800e0 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1b02a0 con 0x55df63074f00 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1af500 con 0x55df63075820 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 10 client.0 _ll_register_callbacks cb 0x7f9434031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944110c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:47454/0 (socket says 172.21.15.37:47454) 2024-04-26T18:27:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 -- 172.21.15.37:0/2060253321 learned_addr learned my addr 172.21.15.37:0/2060253321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944210e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df6307aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 -- 172.21.15.37:0/2060253321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 msgr2=0x55df6307a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 -- 172.21.15.37:0/2060253321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 msgr2=0x55df6307aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df6307aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 -- 172.21.15.37:0/2060253321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df630f5a30 con 0x55df63074f00 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f944190d640 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f943809c6b0 tx=0x7f943804aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d727f55fa42204e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f94257fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943801c040 con 0x55df63074f00 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f943809aa20 con 0x55df63074f00 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.236+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f942c021070 con 0x55df63074f00 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f943804d040 con 0x55df63074f00 2024-04-26T18:27:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 10 client.25268 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df630f6d80 con 0x55df63074f00 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f942c021070 con 0x55df63074f00 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f9438098d90 con 0x55df63074f00 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 10 client.25268 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 20 client.25268 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630f70e0 0x55df630f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 4 client.25268 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df62fe2ee0 con 0x55df630f70e0 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.237+0000 7f944110c640 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630f70e0 0x55df630f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.238+0000 7f944110c640 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630f70e0 0x55df630f94d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8b3a372b0d0dad server_cookie=bf3e6d633a085a9f in_seq=0 out_seq=0 2024-04-26T18:27:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.238+0000 7f94257fa640 10 client.25268 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f94257fa640 1 -- 172.21.15.37:0/2060253321 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df62fe2ee0 con 0x55df630f70e0 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f94257fa640 10 client.25268 handle_command_reply: tid=0 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 client.25268 shutdown 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 msgr2=0x55df6307a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f943809c6b0 tx=0x7f943804aad0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 shutdown_connections 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55df630f70e0 0x55df630f94d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df630800e0 0x55df6307aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f94257fa640 0 client.25268 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55df63075820 0x55df6307a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 --2- 172.21.15.37:0/2060253321 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55df63074f00 0x55df6307a000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.249+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 >> 172.21.15.37:0/2060253321 conn(0x55df63071410 msgr2=0x55df6307d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 shutdown_connections 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9446966740 1 -- 172.21.15.37:0/2060253321 wait complete. 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9446966740 20 client.25268 trim_cache size 0 max 16384 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:27:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:27:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:27:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:27:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 msgr2=0x7f94280a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 0x7f94280a62b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9434009a60 tx=0x7f94340099f0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 msgr2=0x7f943c074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f943800e9f0 tx=0x7f943800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 shutdown_connections 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f94280a3df0 0x7f94280a62b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f943c103300 0x7f943c075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f943c108950 0x7f943c0756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 --2- 172.21.15.37:0/2927799708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f943c102940 0x7f943c074c30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 >> 172.21.15.37:0/2927799708 conn(0x7f943c079dc0 msgr2=0x7f943c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 shutdown_connections 2024-04-26T18:27:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:00.250+0000 7f9443b71640 1 -- 172.21.15.37:0/2927799708 wait complete. 2024-04-26T18:27:00.265 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T18:27:00.265 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T18:27:00.265 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T18:27:00.266 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gmake... /usr/bin/gmake 2024-04-26T18:27:00.266 INFO:tasks.workunit.client.0.smithi037.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-26T18:27:00.267 INFO:tasks.workunit.client.0.smithi037.stdout:checking for tar... /bin/tar 2024-04-26T18:27:00.267 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gzip... /bin/gzip 2024-04-26T18:27:00.307 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether gcc -MM is supported... yes 2024-04-26T18:27:00.307 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sort... /bin/sort 2024-04-26T18:27:00.308 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether ln -s works... yes 2024-04-26T18:27:00.308 INFO:tasks.workunit.client.0.smithi037.stdout:checking for rpm... /bin/rpm 2024-04-26T18:27:00.315 INFO:tasks.workunit.client.0.smithi037.stdout:checking for rpmbuild... no 2024-04-26T18:27:00.371 INFO:tasks.workunit.client.0.smithi037.stdout:checking assert.h usability... yes 2024-04-26T18:27:00.389 INFO:tasks.workunit.client.0.smithi037.stdout:checking assert.h presence... yes 2024-04-26T18:27:00.389 INFO:tasks.workunit.client.0.smithi037.stdout:checking for assert.h... yes 2024-04-26T18:27:00.453 INFO:tasks.workunit.client.0.smithi037.stdout:checking bstring.h usability... no 2024-04-26T18:27:00.486 INFO:tasks.workunit.client.0.smithi037.stdout:checking bstring.h presence... no 2024-04-26T18:27:00.486 INFO:tasks.workunit.client.0.smithi037.stdout:checking for bstring.h... no 2024-04-26T18:27:00.548 INFO:tasks.workunit.client.0.smithi037.stdout:checking libgen.h usability... yes 2024-04-26T18:27:00.566 INFO:tasks.workunit.client.0.smithi037.stdout:checking libgen.h presence... yes 2024-04-26T18:27:00.566 INFO:tasks.workunit.client.0.smithi037.stdout:checking for libgen.h... yes 2024-04-26T18:27:00.636 INFO:tasks.workunit.client.0.smithi037.stdout:checking dirent.h usability... yes 2024-04-26T18:27:00.655 INFO:tasks.workunit.client.0.smithi037.stdout:checking dirent.h presence... yes 2024-04-26T18:27:00.655 INFO:tasks.workunit.client.0.smithi037.stdout:checking for dirent.h... yes 2024-04-26T18:27:00.715 INFO:tasks.workunit.client.0.smithi037.stdout:checking errno.h usability... yes 2024-04-26T18:27:00.735 INFO:tasks.workunit.client.0.smithi037.stdout:checking errno.h presence... yes 2024-04-26T18:27:00.735 INFO:tasks.workunit.client.0.smithi037.stdout:checking for errno.h... yes 2024-04-26T18:27:00.793 INFO:tasks.workunit.client.0.smithi037.stdout:checking malloc.h usability... yes 2024-04-26T18:27:00.813 INFO:tasks.workunit.client.0.smithi037.stdout:checking malloc.h presence... yes 2024-04-26T18:27:00.813 INFO:tasks.workunit.client.0.smithi037.stdout:checking for malloc.h... yes 2024-04-26T18:27:00.875 INFO:tasks.workunit.client.0.smithi037.stdout:checking uuid.h usability... no 2024-04-26T18:27:00.906 INFO:tasks.workunit.client.0.smithi037.stdout:checking uuid.h presence... no 2024-04-26T18:27:00.906 INFO:tasks.workunit.client.0.smithi037.stdout:checking for uuid.h... no 2024-04-26T18:27:00.963 INFO:tasks.workunit.client.0.smithi037.stdout:checking uuid/uuid.h usability... yes 2024-04-26T18:27:00.983 INFO:tasks.workunit.client.0.smithi037.stdout:checking uuid/uuid.h presence... yes 2024-04-26T18:27:00.984 INFO:tasks.workunit.client.0.smithi037.stdout:checking for uuid/uuid.h... yes 2024-04-26T18:27:01.045 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/uuid.h usability... no 2024-04-26T18:27:01.076 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/uuid.h presence... no 2024-04-26T18:27:01.076 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/uuid.h... no 2024-04-26T18:27:01.138 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/file.h usability... yes 2024-04-26T18:27:01.157 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/file.h presence... yes 2024-04-26T18:27:01.157 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/file.h... yes 2024-04-26T18:27:01.216 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fcntl.h usability... yes 2024-04-26T18:27:01.238 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fcntl.h presence... yes 2024-04-26T18:27:01.238 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/fcntl.h... yes 2024-04-26T18:27:01.299 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/syssgi.h usability... no 2024-04-26T18:27:01.331 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/syssgi.h presence... no 2024-04-26T18:27:01.331 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/syssgi.h... no 2024-04-26T18:27:01.409 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/param.h usability... yes 2024-04-26T18:27:01.429 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/param.h presence... yes 2024-04-26T18:27:01.430 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/param.h... yes 2024-04-26T18:27:01.446 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/stat.h... (cached) yes 2024-04-26T18:27:01.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:01 smithi037 ceph-mon[29171]: pgmap v83: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:27:01.492 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/statvfs.h usability... yes 2024-04-26T18:27:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:01 smithi120 ceph-mon[32244]: pgmap v83: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:27:01.511 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/statvfs.h presence... yes 2024-04-26T18:27:01.511 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/statvfs.h... yes 2024-04-26T18:27:01.572 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/time.h usability... yes 2024-04-26T18:27:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:01 smithi170 ceph-mon[32697]: pgmap v83: 97 pgs: 97 active+clean; 41 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-04-26T18:27:01.591 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/time.h presence... yes 2024-04-26T18:27:01.592 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/time.h... yes 2024-04-26T18:27:01.651 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/ioctl.h usability... yes 2024-04-26T18:27:01.670 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/ioctl.h presence... yes 2024-04-26T18:27:01.670 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/ioctl.h... yes 2024-04-26T18:27:01.741 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/wait.h usability... yes 2024-04-26T18:27:01.762 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/wait.h presence... yes 2024-04-26T18:27:01.763 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/wait.h... yes 2024-04-26T18:27:01.779 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/types.h... (cached) yes 2024-04-26T18:27:01.784 INFO:tasks.workunit.client.0.smithi037.stdout:checking for strings.h... (cached) yes 2024-04-26T18:27:01.830 INFO:tasks.workunit.client.0.smithi037.stdout:checking err.h usability... yes 2024-04-26T18:27:01.849 INFO:tasks.workunit.client.0.smithi037.stdout:checking err.h presence... yes 2024-04-26T18:27:01.849 INFO:tasks.workunit.client.0.smithi037.stdout:checking for err.h... yes 2024-04-26T18:27:01.906 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/falloc.h usability... yes 2024-04-26T18:27:01.924 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/falloc.h presence... yes 2024-04-26T18:27:01.924 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/falloc.h... yes 2024-04-26T18:27:01.989 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-26T18:27:02.021 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-26T18:27:02.021 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-26T18:27:02.076 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-26T18:27:02.107 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-26T18:27:02.107 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-26T18:27:02.163 INFO:tasks.workunit.client.0.smithi037.stdout:checking xfs/platform_defs.h usability... no 2024-04-26T18:27:02.193 INFO:tasks.workunit.client.0.smithi037.stdout:checking xfs/platform_defs.h presence... no 2024-04-26T18:27:02.194 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/platform_defs.h... no 2024-04-26T18:27:02.249 INFO:tasks.workunit.client.0.smithi037.stdout:checking btrfs/ioctl.h usability... no 2024-04-26T18:27:02.280 INFO:tasks.workunit.client.0.smithi037.stdout:checking btrfs/ioctl.h presence... no 2024-04-26T18:27:02.281 INFO:tasks.workunit.client.0.smithi037.stdout:checking for btrfs/ioctl.h... no 2024-04-26T18:27:02.339 INFO:tasks.workunit.client.0.smithi037.stdout:checking cifs/ioctl.h usability... no 2024-04-26T18:27:02.376 INFO:tasks.workunit.client.0.smithi037.stdout:checking cifs/ioctl.h presence... no 2024-04-26T18:27:02.377 INFO:tasks.workunit.client.0.smithi037.stdout:checking for cifs/ioctl.h... no 2024-04-26T18:27:02.470 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/mman.h usability... yes 2024-04-26T18:27:02.492 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/mman.h presence... yes 2024-04-26T18:27:02.492 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/mman.h... yes 2024-04-26T18:27:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/ext4.h usability... no 2024-04-26T18:27:02.586 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/ext4.h presence... no 2024-04-26T18:27:02.586 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/ext4.h... no 2024-04-26T18:27:02.624 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/xfs_log_format.h... no 2024-04-26T18:27:02.688 INFO:tasks.workunit.client.0.smithi037.stdout:checking for uuid_compare... no 2024-04-26T18:27:02.737 INFO:tasks.workunit.client.0.smithi037.stdout:checking for uuid_compare in -luuid... yes 2024-04-26T18:27:02.821 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/xfs.h... yes 2024-04-26T18:27:02.858 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/libxfs.h... no 2024-04-26T18:27:02.898 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-26T18:27:02.946 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/xqm.h... yes 2024-04-26T18:27:03.012 INFO:tasks.workunit.client.0.smithi037.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-26T18:27:03.062 INFO:tasks.workunit.client.0.smithi037.stdout:checking xfs/handle.h usability... yes 2024-04-26T18:27:03.079 INFO:tasks.workunit.client.0.smithi037.stdout:checking xfs/handle.h presence... yes 2024-04-26T18:27:03.082 INFO:tasks.workunit.client.0.smithi037.stdout:checking for xfs/handle.h... yes 2024-04-26T18:27:03.135 INFO:tasks.workunit.client.0.smithi037.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-26T18:27:03.182 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/acl.h usability... yes 2024-04-26T18:27:03.203 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/acl.h presence... yes 2024-04-26T18:27:03.203 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/acl.h... yes 2024-04-26T18:27:03.257 INFO:tasks.workunit.client.0.smithi037.stdout:checking acl/libacl.h usability... yes 2024-04-26T18:27:03.277 INFO:tasks.workunit.client.0.smithi037.stdout:checking acl/libacl.h presence... yes 2024-04-26T18:27:03.277 INFO:tasks.workunit.client.0.smithi037.stdout:checking for acl/libacl.h... yes 2024-04-26T18:27:03.333 INFO:tasks.workunit.client.0.smithi037.stdout:checking for acl_init in -lacl... yes 2024-04-26T18:27:03.380 INFO:tasks.workunit.client.0.smithi037.stdout:checking gdbm-ndbm.h usability... no 2024-04-26T18:27:03.410 INFO:tasks.workunit.client.0.smithi037.stdout:checking gdbm-ndbm.h presence... no 2024-04-26T18:27:03.410 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gdbm-ndbm.h... no 2024-04-26T18:27:03.457 INFO:tasks.workunit.client.0.smithi037.stdout:checking gdbm/ndbm.h usability... no 2024-04-26T18:27:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:03 smithi037 ceph-mon[29171]: pgmap v84: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 392 op/s 2024-04-26T18:27:03.487 INFO:tasks.workunit.client.0.smithi037.stdout:checking gdbm/ndbm.h presence... no 2024-04-26T18:27:03.487 INFO:tasks.workunit.client.0.smithi037.stdout:checking for gdbm/ndbm.h... no 2024-04-26T18:27:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:03 smithi120 ceph-mon[32244]: pgmap v84: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 392 op/s 2024-04-26T18:27:03.532 INFO:tasks.workunit.client.0.smithi037.stdout:checking ndbm.h usability... no 2024-04-26T18:27:03.562 INFO:tasks.workunit.client.0.smithi037.stdout:checking ndbm.h presence... no 2024-04-26T18:27:03.563 INFO:tasks.workunit.client.0.smithi037.stdout:checking for ndbm.h... no 2024-04-26T18:27:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:03 smithi170 ceph-mon[32697]: pgmap v84: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 392 op/s 2024-04-26T18:27:03.607 INFO:tasks.workunit.client.0.smithi037.stdout:checking libaio.h usability... yes 2024-04-26T18:27:03.628 INFO:tasks.workunit.client.0.smithi037.stdout:checking libaio.h presence... yes 2024-04-26T18:27:03.628 INFO:tasks.workunit.client.0.smithi037.stdout:checking for libaio.h... yes 2024-04-26T18:27:03.679 INFO:tasks.workunit.client.0.smithi037.stdout:checking liburing.h usability... no 2024-04-26T18:27:03.710 INFO:tasks.workunit.client.0.smithi037.stdout:checking liburing.h presence... no 2024-04-26T18:27:03.710 INFO:tasks.workunit.client.0.smithi037.stdout:checking for liburing.h... no 2024-04-26T18:27:03.757 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/capability.h usability... no 2024-04-26T18:27:03.788 INFO:tasks.workunit.client.0.smithi037.stdout:checking sys/capability.h presence... no 2024-04-26T18:27:03.788 INFO:tasks.workunit.client.0.smithi037.stdout:checking for sys/capability.h... no 2024-04-26T18:27:03.832 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/fiemap.h usability... yes 2024-04-26T18:27:03.850 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/fiemap.h presence... yes 2024-04-26T18:27:03.850 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/fiemap.h... yes 2024-04-26T18:27:03.901 INFO:tasks.workunit.client.0.smithi037.stdout:checking for fallocate... yes 2024-04-26T18:27:03.963 INFO:tasks.workunit.client.0.smithi037.stdout:checking for open_by_handle_at... yes 2024-04-26T18:27:04.014 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/fs.h usability... yes 2024-04-26T18:27:04.033 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/fs.h presence... yes 2024-04-26T18:27:04.033 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/fs.h... yes 2024-04-26T18:27:04.083 INFO:tasks.workunit.client.0.smithi037.stdout:checking btrfsutil.h usability... no 2024-04-26T18:27:04.119 INFO:tasks.workunit.client.0.smithi037.stdout:checking btrfsutil.h presence... no 2024-04-26T18:27:04.120 INFO:tasks.workunit.client.0.smithi037.stdout:checking for btrfsutil.h... no 2024-04-26T18:27:04.185 INFO:tasks.workunit.client.0.smithi037.stdout:checking for copy_file_range... yes 2024-04-26T18:27:04.244 INFO:tasks.workunit.client.0.smithi037.stdout:checking for SEEK_DATA... yes 2024-04-26T18:27:04.316 INFO:tasks.workunit.client.0.smithi037.stdout:checking for BMV_OF_SHARED... yes 2024-04-26T18:27:04.375 INFO:tasks.workunit.client.0.smithi037.stdout:checking for nftw... yes 2024-04-26T18:27:04.435 INFO:tasks.workunit.client.0.smithi037.stdout:checking for RLIMIT_NOFILE... yes 2024-04-26T18:27:04.493 INFO:tasks.workunit.client.0.smithi037.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-26T18:27:04.548 INFO:tasks.workunit.client.0.smithi037.stdout:checking for FICLONE... yes 2024-04-26T18:27:04.606 INFO:tasks.workunit.client.0.smithi037.stdout:checking for renameat2... yes 2024-04-26T18:27:04.661 INFO:tasks.workunit.client.0.smithi037.stdout:checking for reallocarray... yes 2024-04-26T18:27:04.731 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct mount_attr... yes 2024-04-26T18:27:04.806 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-26T18:27:04.878 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-26T18:27:04.951 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-26T18:27:05.022 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-26T18:27:05.093 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-26T18:27:05.166 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-26T18:27:05.237 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-26T18:27:05.290 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/btrfs.h usability... yes 2024-04-26T18:27:05.309 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/btrfs.h presence... yes 2024-04-26T18:27:05.309 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/btrfs.h... yes 2024-04-26T18:27:05.368 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-26T18:27:05.388 INFO:tasks.workunit.client.0.smithi037.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-26T18:27:05.388 INFO:tasks.workunit.client.0.smithi037.stdout:checking for linux/btrfs_tree.h... yes 2024-04-26T18:27:05.441 INFO:tasks.workunit.client.0.smithi037.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-26T18:27:05.481 INFO:tasks.workunit.client.0.smithi037.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-26T18:27:05.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:05 smithi037 ceph-mon[29171]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.9 MiB/s wr, 383 op/s 2024-04-26T18:27:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:05 smithi120 ceph-mon[32244]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.9 MiB/s wr, 383 op/s 2024-04-26T18:27:05.506 INFO:tasks.workunit.client.0.smithi037.stdout:configure: creating ./config.status 2024-04-26T18:27:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:05 smithi170 ceph-mon[32697]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 1.9 MiB/s wr, 383 op/s 2024-04-26T18:27:05.869 INFO:tasks.workunit.client.0.smithi037.stdout:config.status: creating include/builddefs 2024-04-26T18:27:05.882 INFO:tasks.workunit.client.0.smithi037.stdout:config.status: creating include/config.h 2024-04-26T18:27:05.894 INFO:tasks.workunit.client.0.smithi037.stdout:config.status: executing libtool commands 2024-04-26T18:27:05.984 INFO:tasks.workunit.client.0.smithi037.stdout:Building include 2024-04-26T18:27:05.985 INFO:tasks.workunit.client.0.smithi037.stdout:Building lib 2024-04-26T18:27:05.985 INFO:tasks.workunit.client.0.smithi037.stdout:Building tests 2024-04-26T18:27:05.989 INFO:tasks.workunit.client.0.smithi037.stdout: [LTDEP] 2024-04-26T18:27:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:Building btrfs 2024-04-26T18:27:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:Building ceph 2024-04-26T18:27:05.992 INFO:tasks.workunit.client.0.smithi037.stdout:Building cifs 2024-04-26T18:27:06.000 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-26T18:27:06.002 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-26T18:27:06.018 INFO:tasks.workunit.client.0.smithi037.stdout:Building ext4 2024-04-26T18:27:06.021 INFO:tasks.workunit.client.0.smithi037.stdout:Building f2fs 2024-04-26T18:27:06.022 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-26T18:27:06.028 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-26T18:27:06.033 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-26T18:27:06.051 INFO:tasks.workunit.client.0.smithi037.stdout:Building generic 2024-04-26T18:27:06.089 INFO:tasks.workunit.client.0.smithi037.stdout:Building nfs 2024-04-26T18:27:06.096 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-26T18:27:06.098 INFO:tasks.workunit.client.0.smithi037.stdout:Building ocfs2 2024-04-26T18:27:06.106 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-26T18:27:06.115 INFO:tasks.workunit.client.0.smithi037.stdout:Building overlay 2024-04-26T18:27:06.124 INFO:tasks.workunit.client.0.smithi037.stdout:Building perf 2024-04-26T18:27:06.134 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-26T18:27:06.148 INFO:tasks.workunit.client.0.smithi037.stdout:Building selftest 2024-04-26T18:27:06.153 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-26T18:27:06.155 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-26T18:27:06.173 INFO:tasks.workunit.client.0.smithi037.stdout:Building shared 2024-04-26T18:27:06.180 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-26T18:27:06.201 INFO:tasks.workunit.client.0.smithi037.stdout:Building tmpfs 2024-04-26T18:27:06.202 INFO:tasks.workunit.client.0.smithi037.stdout:Building udf 2024-04-26T18:27:06.207 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-26T18:27:06.207 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-26T18:27:06.208 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-26T18:27:06.209 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dataascii.lo 2024-04-26T18:27:06.224 INFO:tasks.workunit.client.0.smithi037.stdout:Building xfs 2024-04-26T18:27:06.232 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] databin.lo 2024-04-26T18:27:06.332 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] datapid.lo 2024-04-26T18:27:06.359 INFO:tasks.workunit.client.0.smithi037.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-26T18:27:06.363 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] file_lock.lo 2024-04-26T18:27:06.425 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] forker.lo 2024-04-26T18:27:06.479 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] pattern.lo 2024-04-26T18:27:06.482 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] open_flags.lo 2024-04-26T18:27:06.561 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] random_range.lo 2024-04-26T18:27:06.600 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] string_to_tokens.lo 2024-04-26T18:27:06.629 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] str_to_bytes.lo 2024-04-26T18:27:06.658 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] tlibio.lo 2024-04-26T18:27:06.735 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] write_log.lo 2024-04-26T18:27:06.804 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] random.lo 2024-04-26T18:27:07.006 INFO:tasks.workunit.client.0.smithi037.stdout: [LD] 2024-04-26T18:27:07.118 INFO:tasks.workunit.client.0.smithi037.stdout:Building ltp 2024-04-26T18:27:07.119 INFO:tasks.workunit.client.0.smithi037.stdout:Building src 2024-04-26T18:27:07.119 INFO:tasks.workunit.client.0.smithi037.stdout:Building m4 2024-04-26T18:27:07.119 INFO:tasks.workunit.client.0.smithi037.stdout:Building common 2024-04-26T18:27:07.123 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:07.136 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:07.319 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] doio 2024-04-26T18:27:07.319 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fsstress 2024-04-26T18:27:07.319 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fsx 2024-04-26T18:27:07.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:07 smithi037 ceph-mon[29171]: pgmap v86: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 427 op/s 2024-04-26T18:27:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:07 smithi120 ceph-mon[32244]: pgmap v86: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 427 op/s 2024-04-26T18:27:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:07 smithi170 ceph-mon[32697]: pgmap v86: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 427 op/s 2024-04-26T18:27:08.035 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] growfiles 2024-04-26T18:27:08.075 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] iogen 2024-04-26T18:27:08.464 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-stress 2024-04-26T18:27:09.090 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:09.092 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:09.092 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:09.093 INFO:tasks.workunit.client.0.smithi037.stdout: [DEP] 2024-04-26T18:27:09.328 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dirstress 2024-04-26T18:27:09.328 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fill 2024-04-26T18:27:09.328 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fill2 2024-04-26T18:27:09.329 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] getpagesize 2024-04-26T18:27:09.461 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] holes 2024-04-26T18:27:09.478 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] lstat64 2024-04-26T18:27:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:09 smithi037 ceph-mon[29171]: pgmap v87: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:09 smithi120 ceph-mon[32244]: pgmap v87: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:09.556 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] nametest 2024-04-26T18:27:09.563 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] permname 2024-04-26T18:27:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:09 smithi170 ceph-mon[32697]: pgmap v87: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:09.693 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] randholes 2024-04-26T18:27:09.697 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] runas 2024-04-26T18:27:09.770 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] truncfile 2024-04-26T18:27:09.820 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] usemem 2024-04-26T18:27:09.884 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] mmapcat 2024-04-26T18:27:09.965 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] append_reader 2024-04-26T18:27:09.966 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] append_writer 2024-04-26T18:27:09.970 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dirperf 2024-04-26T18:27:10.041 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] metaperf 2024-04-26T18:27:10.122 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] devzero 2024-04-26T18:27:10.124 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] feature 2024-04-26T18:27:10.230 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] alloc 2024-04-26T18:27:10.295 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fault 2024-04-26T18:27:10.391 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fstest 2024-04-26T18:27:10.403 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_access_root 2024-04-26T18:27:10.424 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] godown 2024-04-26T18:27:10.449 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] resvtest 2024-04-26T18:27:10.559 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] writemod 2024-04-26T18:27:10.595 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] writev_on_pagefault 2024-04-26T18:27:10.635 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] makeextents 2024-04-26T18:27:10.636 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] itrash 2024-04-26T18:27:10.701 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] rename 2024-04-26T18:27:10.768 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] multi_open_unlink 2024-04-26T18:27:10.789 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] unwritten_sync 2024-04-26T18:27:10.813 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] genhashnames 2024-04-26T18:27:10.846 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_holes 2024-04-26T18:27:10.949 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_writev 2024-04-26T18:27:10.976 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_truncate_cmtime 2024-04-26T18:27:11.003 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dirhash_collide 2024-04-26T18:27:11.008 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_rename_overwrite 2024-04-26T18:27:11.109 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] holetest 2024-04-26T18:27:11.135 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_truncate_self 2024-04-26T18:27:11.146 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] af_unix 2024-04-26T18:27:11.197 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_stale_pmd 2024-04-26T18:27:11.278 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_cow_race 2024-04-26T18:27:11.294 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_fallocate 2024-04-26T18:27:11.349 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fsync-err 2024-04-26T18:27:11.352 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_write_ro 2024-04-26T18:27:11.451 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-26T18:27:11.466 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-26T18:27:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:11 smithi037 ceph-mon[29171]: pgmap v88: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:11 smithi120 ceph-mon[32244]: pgmap v88: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:11.522 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_ofd_locks 2024-04-26T18:27:11.546 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_collision 2024-04-26T18:27:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:11 smithi170 ceph-mon[32697]: pgmap v88: 97 pgs: 97 active+clean; 44 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 351 KiB/s wr, 73 op/s 2024-04-26T18:27:11.616 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] mmap-write-concurrent 2024-04-26T18:27:11.621 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_get_file_time 2024-04-26T18:27:11.733 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_create_short_dirs 2024-04-26T18:27:11.753 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_create_long_dirs 2024-04-26T18:27:11.773 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_enospc 2024-04-26T18:27:11.798 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_writev_overlap 2024-04-26T18:27:11.901 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] checkpoint_journal 2024-04-26T18:27:11.926 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] mmap-rw-fault 2024-04-26T18:27:11.975 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] allocstale 2024-04-26T18:27:11.976 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_cow_memory_failure 2024-04-26T18:27:12.065 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fake-dump-rootino 2024-04-26T18:27:12.124 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dio-buf-fault 2024-04-26T18:27:12.146 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] rewinddir-test 2024-04-26T18:27:12.170 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] readdir-while-renames 2024-04-26T18:27:12.229 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] xfsctl 2024-04-26T18:27:12.286 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] bstat 2024-04-26T18:27:12.320 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mtab 2024-04-26T18:27:12.342 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] getdevicesize 2024-04-26T18:27:12.686 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] preallo_rw_pattern_reader 2024-04-26T18:27:13.094 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] preallo_rw_pattern_writer 2024-04-26T18:27:13.094 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] ftrunc 2024-04-26T18:27:13.098 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] trunc 2024-04-26T18:27:13.147 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fs_perms 2024-04-26T18:27:13.244 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] testx 2024-04-26T18:27:13.263 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] looptest 2024-04-26T18:27:13.266 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] locktest 2024-04-26T18:27:13.327 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] unwritten_mmap 2024-04-26T18:27:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 -- 172.21.15.37:0/1335213007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654103c00 msgr2=0x7fa654103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 --2- 172.21.15.37:0/1335213007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654103c00 0x7fa654103fe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa64402a480 comp rx=0 tx=0).stop 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 -- 172.21.15.37:0/1335213007 shutdown_connections 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 --2- 172.21.15.37:0/1335213007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa65410cf40 0x7fa65410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 --2- 172.21.15.37:0/1335213007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa654104520 0x7fa65410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 --2- 172.21.15.37:0/1335213007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654103c00 0x7fa654103fe0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 -- 172.21.15.37:0/1335213007 >> 172.21.15.37:0/1335213007 conn(0x7fa654078980 msgr2=0x7fa654078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 -- 172.21.15.37:0/1335213007 shutdown_connections 2024-04-26T18:27:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.370+0000 7fa658dc6640 1 -- 172.21.15.37:0/1335213007 wait complete. 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.371+0000 7fa658dc6640 1 Processor -- start 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.371+0000 7fa658dc6640 1 -- start start 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654104520 0x7fa654199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65410cf40 0x7fa65419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa654112c20 con 0x7fa654104520 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa654112aa0 con 0x7fa654103c00 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa658dc6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa654112da0 con 0x7fa65410cf40 2024-04-26T18:27:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654104520 0x7fa654199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:46432/0 (socket says 172.21.15.37:46432) 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 -- 172.21.15.37:0/137432299 learned_addr learned my addr 172.21.15.37:0/137432299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa652d9d640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65410cf40 0x7fa65419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 -- 172.21.15.37:0/137432299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65410cf40 msgr2=0x7fa65419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65410cf40 0x7fa65419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 -- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654104520 msgr2=0x7fa654199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654104520 0x7fa654199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.372+0000 7fa65259c640 1 -- 172.21.15.37:0/137432299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65419e0a0 con 0x7fa654103c00 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa65259c640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa644005ab0 tx=0x7fa644003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa6437fe640 1 -- 172.21.15.37:0/137432299 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64403f070 con 0x7fa654103c00 2024-04-26T18:27:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65419e330 con 0x7fa654103c00 2024-04-26T18:27:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa654101780 con 0x7fa654103c00 2024-04-26T18:27:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa6437fe640 1 -- 172.21.15.37:0/137432299 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa644031d60 con 0x7fa654103c00 2024-04-26T18:27:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.373+0000 7fa6437fe640 1 -- 172.21.15.37:0/137432299 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644035030 con 0x7fa654103c00 2024-04-26T18:27:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.376+0000 7fa65bbc1740 1 Processor -- start 2024-04-26T18:27:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.376+0000 7fa65bbc1740 1 -- start start 2024-04-26T18:27:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa6437fe640 1 -- 172.21.15.37:0/137432299 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa644042070 con 0x7fa654103c00 2024-04-26T18:27:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x5569812fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x556981306ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556981262460 con 0x5569813070e0 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55698121c590 con 0x5569812fbf00 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55698125e420 con 0x5569812fc820 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa651d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39080/0 (socket says 172.21.15.37:39080) 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa651d9b640 1 -- 172.21.15.37:0/1801807852 learned_addr learned my addr 172.21.15.37:0/1801807852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa652d9d640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x5569812fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa65259c640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x556981306ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.377+0000 7fa651d9b640 1 -- 172.21.15.37:0/1801807852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 msgr2=0x556981306ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa651d9b640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x556981306ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa651d9b640 1 -- 172.21.15.37:0/1801807852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 msgr2=0x5569812fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa651d9b640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x5569812fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa651d9b640 1 -- 172.21.15.37:0/1801807852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569812290d0 con 0x5569813070e0 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa65259c640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x556981306ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa652d9d640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x5569812fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa6437fe640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 0x7fa63c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa652d9d640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 0x7fa63c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa6437fe640 1 -- 172.21.15.37:0/137432299 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa6440f0e40 con 0x7fa654103c00 2024-04-26T18:27:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.378+0000 7fa651d9b640 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa64800ef70 tx=0x7fa64800e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797d65e649662cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa640ff9640 1 -- 172.21.15.37:0/1801807852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64800e2d0 con 0x5569813070e0 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa640ff9640 1 -- 172.21.15.37:0/1801807852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64801d070 con 0x5569813070e0 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa640ff9640 1 -- 172.21.15.37:0/1801807852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480187f0 con 0x5569813070e0 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa652d9d640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 0x7fa63c0a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c009500 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1801807852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 msgr2=0x5569813094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.379+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa64800ef70 tx=0x7fa64800e660 comp rx=0 tx=0).stop 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1801807852 shutdown_connections 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x5569813094d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x556981306ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1801807852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x5569812fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1801807852 >> 172.21.15.37:0/1801807852 conn(0x5569812f8410 msgr2=0x5569812f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1801807852 shutdown_connections 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1801807852 wait complete. 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 -- start start 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.380+0000 7fa65bbc1740 1 Processor -- start 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x5569813044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x556981304a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556981309cb0 con 0x5569813070e0 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5569811e9b60 con 0x5569812fbf00 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556981309e30 con 0x5569812fc820 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65bbc1740 10 client.0 _ll_register_callbacks cb 0x55698120e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x556981304a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:46450/0 (socket says 172.21.15.37:46450) 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 -- 172.21.15.37:0/547100436 learned_addr learned my addr 172.21.15.37:0/547100436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa651d9b640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x5569813044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 -- 172.21.15.37:0/547100436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 msgr2=0x5569813044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x5569813044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 -- 172.21.15.37:0/547100436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 msgr2=0x556981304a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x556981304a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.381+0000 7fa65259c640 1 -- 172.21.15.37:0/547100436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556980f74210 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa65259c640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6440bcd60 tx=0x7fa644031f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb72089429685d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644049070 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa6440bd030 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64801d070 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6440bc910 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa65bbc1740 10 client.25280 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556981040620 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.382+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64801d070 con 0x5569812fbf00 2024-04-26T18:27:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa6440f2030 con 0x5569812fbf00 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 10 client.25280 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 20 client.25280 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556981317370 0x556981319810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 4 client.25280 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556981270070 con 0x556981317370 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa651d9b640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556981317370 0x556981319810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa639ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.383+0000 7fa651d9b640 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556981317370 0x556981319810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511f5043f9215977 server_cookie=d29b7d3b720b0e38 in_seq=0 out_seq=0 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa639ffb640 1 -- 172.21.15.37:0/547100436 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556981270070 con 0x556981317370 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa639ffb640 10 client.25280 handle_command_reply: tid=0 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 client.25280 shutdown 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 msgr2=0x556981303f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6440bcd60 tx=0x7fa644031f10 comp rx=0 tx=0).stop 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 shutdown_connections 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556981317370 0x556981319810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569813070e0 0x556981304a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569812fc820 0x5569813044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/547100436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fbf00 0x556981303f90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 >> 172.21.15.37:0/547100436 conn(0x5569812f8410 msgr2=0x5569812fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa639ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.384+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 shutdown_connections 2024-04-26T18:27:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.385+0000 7fa65bbc1740 1 -- 172.21.15.37:0/547100436 wait complete. 2024-04-26T18:27:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.385+0000 7fa65bbc1740 20 client.25280 trim_cache size 0 max 16384 2024-04-26T18:27:13.392 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] bulkstat_unlink_test 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 Processor -- start 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 -- start start 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x5569812fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x556981303870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa634010ff0 con 0x5569812fbf00 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa63400e8c0 con 0x5569812fc820 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa63400c710 con 0x5569813070e0 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa652d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa652d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:33924/0 (socket says 172.21.15.37:33924) 2024-04-26T18:27:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa652d9d640 1 -- 172.21.15.37:0/1418992629 learned_addr learned my addr 172.21.15.37:0/1418992629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa65259c640 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x5569812fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.404+0000 7fa651d9b640 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x556981303870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 -- 172.21.15.37:0/1418992629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 msgr2=0x556981303870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x556981303870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 -- 172.21.15.37:0/1418992629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 msgr2=0x5569812fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x5569812fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 -- 172.21.15.37:0/1418992629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569812fc650 con 0x5569813070e0 2024-04-26T18:27:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa652d9d640 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005c00 tx=0x7fa64c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420f17aa517a1f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa63bfff640 1 -- 172.21.15.37:0/1418992629 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011500 con 0x5569813070e0 2024-04-26T18:27:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa63bfff640 1 -- 172.21.15.37:0/1418992629 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64c011b50 con 0x5569813070e0 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.405+0000 7fa63bfff640 1 -- 172.21.15.37:0/1418992629 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c02ad10 con 0x5569813070e0 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1418992629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 msgr2=0x5569812f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005c00 tx=0x7fa64c01a000 comp rx=0 tx=0).stop 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1418992629 shutdown_connections 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x556981303870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569812f6d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/1418992629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x5569812fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.406+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1418992629 >> 172.21.15.37:0/1418992629 conn(0x5569812f8410 msgr2=0x5569813047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1418992629 shutdown_connections 2024-04-26T18:27:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- 172.21.15.37:0/1418992629 wait complete. 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- start start 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 Processor -- start 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x5569813084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569813089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa63400cd80 con 0x5569812fbf00 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa63400e1f0 con 0x5569812fc820 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa63400d450 con 0x5569813070e0 2024-04-26T18:27:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65bbc1740 10 client.0 _ll_register_callbacks cb 0x7fa644005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39106/0 (socket says 172.21.15.37:39106) 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.407+0000 7fa65259c640 1 -- 172.21.15.37:0/250636266 learned_addr learned my addr 172.21.15.37:0/250636266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa652d9d640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569813089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa651d9b640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x5569813084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 -- 172.21.15.37:0/250636266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 msgr2=0x5569813089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569813089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 -- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 msgr2=0x5569813084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x5569813084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 -- 172.21.15.37:0/250636266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698130d300 con 0x5569812fbf00 2024-04-26T18:27:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65259c640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa6440bb080 tx=0x7fa644005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbd60892125bb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644049040 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa6440c57b0 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.408+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64c011b50 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6440bc730 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 10 client.15660 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55698130d300 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64c011b50 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa6440bc220 con 0x5569812fbf00 2024-04-26T18:27:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 20 client.15660 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55698137e080 0x5569813804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 4 client.15660 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.409+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556981269ee0 con 0x55698137e080 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.410+0000 7fa651d9b640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55698137e080 0x5569813804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.410+0000 7fa639ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.410+0000 7fa651d9b640 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55698137e080 0x5569813804a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad61885d06fbc0eb server_cookie=5632d64e6e5d7d35 in_seq=0 out_seq=0 2024-04-26T18:27:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:13 smithi037 ceph-mon[29171]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 393 KiB/s wr, 85 op/s 2024-04-26T18:27:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:13 smithi120 ceph-mon[32244]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 393 KiB/s wr, 85 op/s 2024-04-26T18:27:13.503 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] deduperace 2024-04-26T18:27:13.516 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] bulkstat_unlink_test_modified 2024-04-26T18:27:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:13 smithi170 ceph-mon[32697]: pgmap v89: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 393 KiB/s wr, 85 op/s 2024-04-26T18:27:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.581+0000 7fa639ffb640 1 -- 172.21.15.37:0/250636266 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556981269ee0 con 0x55698137e080 2024-04-26T18:27:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.581+0000 7fa639ffb640 10 client.15660 handle_command_reply: tid=0 2024-04-26T18:27:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.581+0000 7fa65bbc1740 1 client.15660 shutdown 2024-04-26T18:27:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.581+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 msgr2=0x556981307f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.581+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa6440bb080 tx=0x7fa644005e00 comp rx=0 tx=0).stop 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 shutdown_connections 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55698137e080 0x5569813804a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5569812fc820 0x5569813084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa639ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5569813070e0 0x5569813089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 --2- 172.21.15.37:0/250636266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569812fbf00 0x556981307f70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 >> 172.21.15.37:0/250636266 conn(0x5569812f8410 msgr2=0x5569813057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 shutdown_connections 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 1 -- 172.21.15.37:0/250636266 wait complete. 2024-04-26T18:27:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa65bbc1740 20 client.15660 trim_cache size 0 max 16384 2024-04-26T18:27:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:27:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:27:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:27:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 msgr2=0x7fa63c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.582+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 0x7fa63c0a6300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa64c005fd0 tx=0x7fa64c009500 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 msgr2=0x7fa654199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa644005ab0 tx=0x7fa644003c80 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 shutdown_connections 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa63c0a3e40 0x7fa63c0a6300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa65410cf40 0x7fa65419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa654104520 0x7fa654199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 --2- 172.21.15.37:0/137432299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa654103c00 0x7fa654199100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 >> 172.21.15.37:0/137432299 conn(0x7fa654078980 msgr2=0x7fa6540fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 shutdown_connections 2024-04-26T18:27:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:13.583+0000 7fa658dc6640 1 -- 172.21.15.37:0/137432299 wait complete. 2024-04-26T18:27:13.606 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_dir_offset 2024-04-26T18:27:13.659 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_futimens 2024-04-26T18:27:13.712 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_immutable 2024-04-26T18:27:13.720 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] stale_handle 2024-04-26T18:27:13.757 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] pwrite_mmap_blocked 2024-04-26T18:27:13.809 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_dir_offset2 2024-04-26T18:27:13.896 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] seek_sanity_test 2024-04-26T18:27:13.904 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] seek_copy_test 2024-04-26T18:27:14.012 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_readdir_1 2024-04-26T18:27:14.094 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_readdir_2 2024-04-26T18:27:14.185 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fsync-tester 2024-04-26T18:27:14.304 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] nsexec 2024-04-26T18:27:14.414 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] cloner 2024-04-26T18:27:14.482 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] renameat2 2024-04-26T18:27:14.492 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_getcwd 2024-04-26T18:27:14.499 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] e4compact 2024-04-26T18:27:14.607 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] test-nextquota 2024-04-26T18:27:14.652 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] punch-alternating 2024-04-26T18:27:14.654 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-26T18:27:14.697 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] listxattr 2024-04-26T18:27:14.785 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dio-interleaved 2024-04-26T18:27:14.825 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_dir_type 2024-04-26T18:27:14.837 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] dio-invalidate-cache 2024-04-26T18:27:14.846 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] stat_test 2024-04-26T18:27:14.951 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_encrypted_d_revalidate 2024-04-26T18:27:14.981 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] attr_replace_test 2024-04-26T18:27:15.055 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] swapon 2024-04-26T18:27:15.125 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] mkswap 2024-04-26T18:27:15.132 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_attr_corruption 2024-04-26T18:27:15.168 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_open_tmpfiles 2024-04-26T18:27:15.239 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fscrypt-crypt-util 2024-04-26T18:27:15.301 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] bulkstat_null_ocount 2024-04-26T18:27:15.303 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] splice-test 2024-04-26T18:27:15.363 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] chprojid_fail 2024-04-26T18:27:15.457 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] detached_mounts_propagation 2024-04-26T18:27:15.477 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] ext4_resize 2024-04-26T18:27:15.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:15 smithi037 ceph-mon[29171]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 281 KiB/s wr, 56 op/s 2024-04-26T18:27:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:15 smithi120 ceph-mon[32244]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 281 KiB/s wr, 56 op/s 2024-04-26T18:27:15.522 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_readdir_3 2024-04-26T18:27:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:15 smithi170 ceph-mon[32697]: pgmap v90: 97 pgs: 97 active+clean; 45 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 281 KiB/s wr, 56 op/s 2024-04-26T18:27:15.622 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] splice2pipe 2024-04-26T18:27:15.663 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] uuid_ioctl 2024-04-26T18:27:15.724 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-26T18:27:15.781 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fiemap-fault 2024-04-26T18:27:15.837 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fiemap-tester 2024-04-26T18:27:15.887 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_stripealign 2024-04-26T18:27:15.950 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] open_by_handle 2024-04-26T18:27:16.010 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_mmap_dio 2024-04-26T18:27:16.043 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] xfsfind 2024-04-26T18:27:16.147 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] t_reflink_read_race 2024-04-26T18:27:16.177 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] fssum 2024-04-26T18:27:16.231 INFO:tasks.workunit.client.0.smithi037.stdout:Building log-writes 2024-04-26T18:27:16.237 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] replay-log 2024-04-26T18:27:16.244 INFO:tasks.workunit.client.0.smithi037.stdout:Building perf 2024-04-26T18:27:16.248 INFO:tasks.workunit.client.0.smithi037.stdout:Building aio-dio-regress 2024-04-26T18:27:16.255 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aiocp 2024-04-26T18:27:16.349 INFO:tasks.workunit.client.0.smithi037.stdout:Building vfs 2024-04-26T18:27:16.355 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] vfstest 2024-04-26T18:27:16.478 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-26T18:27:16.653 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-append-write-read-race 2024-04-26T18:27:16.661 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] mount-idmapped 2024-04-26T18:27:16.686 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-cow-race 2024-04-26T18:27:16.834 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-cycle-write 2024-04-26T18:27:16.840 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-eof-race 2024-04-26T18:27:17.012 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-extend-stat 2024-04-26T18:27:17.038 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-fcntl-race 2024-04-26T18:27:17.067 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-hole-filling-race 2024-04-26T18:27:17.174 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-invalidate-failure 2024-04-26T18:27:17.199 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-invalidate-readahead 2024-04-26T18:27:17.218 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aiodio_sparse2 2024-04-26T18:27:17.354 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-subblock-eof-read 2024-04-26T18:27:17.362 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-dio-write-verify 2024-04-26T18:27:17.446 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-26T18:27:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:17 smithi037 ceph-mon[29171]: pgmap v91: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-26T18:27:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:17 smithi120 ceph-mon[32244]: pgmap v91: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-26T18:27:17.529 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-26T18:27:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:17 smithi170 ceph-mon[32697]: pgmap v91: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-26T18:27:17.577 INFO:tasks.workunit.client.0.smithi037.stdout: [CC] aio-last-ref-held-by-io 2024-04-26T18:27:18.415 INFO:tasks.workunit.client.0.smithi037.stderr:+ cd .. 2024-04-26T18:27:18.415 INFO:tasks.workunit.client.0.smithi037.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-26T18:27:18.418 INFO:tasks.workunit.client.0.smithi037.stderr:+ OPTIONS=-z 2024-04-26T18:27:18.418 INFO:tasks.workunit.client.0.smithi037.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-26T18:27:18.418 INFO:tasks.workunit.client.0.smithi037.stdout:Seed set to 1 2024-04-26T18:27:18.440 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T18:27:18.440 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T18:27:18.443 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T18:27:18.443 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T18:27:18.443 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T18:27:18.445 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T18:27:18.447 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero size read 2024-04-26T18:27:18.448 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T18:27:18.448 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero size read 2024-04-26T18:27:18.448 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x81d54 2024-04-26T18:27:18.453 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xb86d7 2024-04-26T18:27:18.461 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xe627e 2024-04-26T18:27:18.500 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0xf3194 2024-04-26T18:27:18.521 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xf3ceb 2024-04-26T18:27:18.527 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0xf47e4 2024-04-26T18:27:18.618 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xfaccc 2024-04-26T18:27:18.978 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xfe423 2024-04-26T18:27:19.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:19 smithi037 ceph-mon[29171]: pgmap v92: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:19 smithi120 ceph-mon[32244]: pgmap v92: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:19 smithi170 ceph-mon[32697]: pgmap v92: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:21.001 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xfe492 2024-04-26T18:27:21.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:21 smithi037 ceph-mon[29171]: pgmap v93: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:21 smithi120 ceph-mon[32244]: pgmap v93: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:21 smithi170 ceph-mon[32697]: pgmap v93: 97 pgs: 97 active+clean; 55 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-04-26T18:27:22.712 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xff038 2024-04-26T18:27:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:23 smithi037 ceph-mon[29171]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-04-26T18:27:23.499 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0xffcac 2024-04-26T18:27:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:23 smithi120 ceph-mon[32244]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-04-26T18:27:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:23 smithi170 ceph-mon[32697]: pgmap v94: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 302 op/s 2024-04-26T18:27:23.602 INFO:tasks.workunit.client.0.smithi037.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-26T18:27:25.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:25 smithi037 ceph-mon[29171]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-04-26T18:27:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:25 smithi120 ceph-mon[32244]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-04-26T18:27:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:25 smithi170 ceph-mon[32697]: pgmap v95: 97 pgs: 97 active+clean; 57 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-04-26T18:27:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 -- 172.21.15.37:0/2994870075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64102930 msgr2=0x7f2a64102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/2994870075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64102930 0x7f2a64102db0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009930 tx=0x7f2a5402a420 comp rx=0 tx=0).stop 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 -- 172.21.15.37:0/2994870075 shutdown_connections 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/2994870075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a641032f0 0x7f2a6410f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/2994870075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64102930 0x7f2a64102db0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/2994870075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a64108940 0x7f2a64108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 -- 172.21.15.37:0/2994870075 >> 172.21.15.37:0/2994870075 conn(0x7f2a640fe400 msgr2=0x7f2a64100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.684+0000 7f2a6bec8640 1 -- 172.21.15.37:0/2994870075 shutdown_connections 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 -- 172.21.15.37:0/2994870075 wait complete. 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 Processor -- start 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 -- start start 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a641032f0 0x7f2a64075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 0x7f2a640756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64112c50 con 0x7f2a64108940 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64112ad0 con 0x7f2a64102930 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a6bec8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64112dd0 con 0x7f2a641032f0 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.685+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a6a465640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 0x7f2a640756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:57954/0 (socket says 172.21.15.37:57954) 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 -- 172.21.15.37:0/4145109534 learned_addr learned my addr 172.21.15.37:0/4145109534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69463640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a641032f0 0x7f2a64075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 -- 172.21.15.37:0/4145109534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a641032f0 msgr2=0x7f2a64075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a641032f0 0x7f2a64075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 -- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 msgr2=0x7f2a640756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 0x7f2a640756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 -- 172.21.15.37:0/4145109534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a75b0 con 0x7f2a64102930 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a6a465640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 0x7f2a640756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a69c64640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9f0 tx=0x7f2a6000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a5affd640 1 -- 172.21.15.37:0/4145109534 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000cd80 con 0x7f2a64102930 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a78a0 con 0x7f2a64102930 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a5affd640 1 -- 172.21.15.37:0/4145109534 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600054d0 con 0x7f2a64102930 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a5affd640 1 -- 172.21.15.37:0/4145109534 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60010690 con 0x7f2a64102930 2024-04-26T18:27:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.686+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a7d70 con 0x7f2a64102930 2024-04-26T18:27:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a5affd640 1 -- 172.21.15.37:0/4145109534 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a60004e50 con 0x7f2a64102930 2024-04-26T18:27:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 Processor -- start 2024-04-26T18:27:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 -- start start 2024-04-26T18:27:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a91698a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a9168e3460 con 0x55a91697cf00 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a91689d590 con 0x55a91697d820 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.689+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a9168df420 con 0x55a9169880e0 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a5affd640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 0x7f2a500a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a6a465640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:57960/0 (socket says 172.21.15.37:57960) 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 -- 172.21.15.37:0/3212703838 learned_addr learned my addr 172.21.15.37:0/3212703838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69463640 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a91698a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a6a465640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 0x7f2a500a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a5affd640 1 -- 172.21.15.37:0/4145109534 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a600d0b50 con 0x7f2a64102930 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 -- 172.21.15.37:0/3212703838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 msgr2=0x55a91698a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a91698a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 -- 172.21.15.37:0/3212703838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 msgr2=0x55a91697d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 -- 172.21.15.37:0/3212703838 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9168aa0d0 con 0x55a91697d820 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a69c64640 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9c0 tx=0x7f2a60004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b348f887aa47f2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a6a465640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 0x7f2a500a62b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0099e0 tx=0x7f2a5c004ab0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a4ffff640 1 -- 172.21.15.37:0/3212703838 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6009c440 con 0x55a91697d820 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a4ffff640 1 -- 172.21.15.37:0/3212703838 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6009d1e0 con 0x55a91697d820 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.690+0000 7f2a4ffff640 1 -- 172.21.15.37:0/3212703838 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600046c0 con 0x55a91697d820 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/3212703838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 msgr2=0x55a916987ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9c0 tx=0x7f2a60004a90 comp rx=0 tx=0).stop 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/3212703838 shutdown_connections 2024-04-26T18:27:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916987ba0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a91698a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/3212703838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/3212703838 >> 172.21.15.37:0/3212703838 conn(0x55a916979410 msgr2=0x55a9169797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/3212703838 shutdown_connections 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/3212703838 wait complete. 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.691+0000 7f2a6ebe4740 1 -- start start 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 Processor -- start 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916985540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a916985a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a91698acb0 con 0x55a91697cf00 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a91686ab60 con 0x55a91697d820 2024-04-26T18:27:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a91698ae30 con 0x55a9169880e0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6ebe4740 10 client.0 _ll_register_callbacks cb 0x55a91688f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43530/0 (socket says 172.21.15.37:43530) 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 -- 172.21.15.37:0/2928839920 learned_addr learned my addr 172.21.15.37:0/2928839920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69463640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916985540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a6a465640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a916985a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 -- 172.21.15.37:0/2928839920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 msgr2=0x55a916985a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a916985a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 -- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 msgr2=0x55a916985540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916985540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.692+0000 7f2a69c64640 1 -- 172.21.15.37:0/2928839920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9165f5210 con 0x55a91697cf00 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a69c64640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce890 tx=0x7f2a6000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea2d76c2f523fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b6020 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a600a3ee0 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6009d1e0 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a6009c440 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 10 client.15672 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9166c1620 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6009d1e0 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a60005140 con 0x55a91697cf00 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 10 client.15672 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 20 client.15672 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a916997ff0 0x55a91699a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 4 client.15672 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.693+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9168f1070 con 0x55a916997ff0 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.694+0000 7f2a69463640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a916997ff0 0x55a91699a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.694+0000 7f2a69463640 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a916997ff0 0x55a91699a3e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218236b751a41b91 server_cookie=9dc312096ca0809d in_seq=0 out_seq=0 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.694+0000 7f2a4d7fa640 10 client.15672 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/2928839920 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9168f1070 con 0x55a916997ff0 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a4d7fa640 10 client.15672 handle_command_reply: tid=0 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 client.15672 shutdown 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 msgr2=0x55a916985000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce890 tx=0x7f2a6000c7d0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 shutdown_connections 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a916997ff0 0x55a91699a3e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697d820 0x55a916985540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a9169880e0 0x55a916985a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/2928839920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a916985000 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 >> 172.21.15.37:0/2928839920 conn(0x55a916979410 msgr2=0x55a91697f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a4d7fa640 0 client.15672 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 shutdown_connections 2024-04-26T18:27:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/2928839920 wait complete. 2024-04-26T18:27:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.695+0000 7f2a6ebe4740 20 client.15672 trim_cache size 0 max 16384 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 Processor -- start 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 -- start start 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a916978430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9169880e0 0x55a91699d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b2fc0 con 0x55a91697cf00 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b0890 con 0x55a9169880e0 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641ae6e0 con 0x55a91697d820 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43534/0 (socket says 172.21.15.37:43534) 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 -- 172.21.15.37:0/257779841 learned_addr learned my addr 172.21.15.37:0/257779841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a6a465640 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9169880e0 0x55a91699d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69463640 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a916978430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 -- 172.21.15.37:0/257779841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 msgr2=0x55a916978430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a916978430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 -- 172.21.15.37:0/257779841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9169880e0 msgr2=0x55a91699d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9169880e0 0x55a91699d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.711+0000 7f2a69c64640 1 -- 172.21.15.37:0/257779841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91697e320 con 0x55a91697cf00 2024-04-26T18:27:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.712+0000 7f2a69c64640 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2a600994b0 tx=0x7f2a6000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421782f622d10780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.712+0000 7f2a4f7fe640 1 -- 172.21.15.37:0/257779841 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b6040 con 0x55a91697cf00 2024-04-26T18:27:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.712+0000 7f2a4f7fe640 1 -- 172.21.15.37:0/257779841 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6009d1e0 con 0x55a91697cf00 2024-04-26T18:27:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.712+0000 7f2a4f7fe640 1 -- 172.21.15.37:0/257779841 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600a3ee0 con 0x55a91697cf00 2024-04-26T18:27:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/257779841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 msgr2=0x55a91697f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2a600994b0 tx=0x7f2a6000c840 comp rx=0 tx=0).stop 2024-04-26T18:27:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/257779841 shutdown_connections 2024-04-26T18:27:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a9169880e0 0x55a91699d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a916978430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/257779841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a91697cf00 0x55a91697f8c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/257779841 >> 172.21.15.37:0/257779841 conn(0x55a916979410 msgr2=0x55a916977d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/257779841 shutdown_connections 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/257779841 wait complete. 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 -- start start 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.713+0000 7f2a6ebe4740 1 Processor -- start 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a91699b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9169880e0 0x55a9169fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641aed50 con 0x55a9169880e0 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b01c0 con 0x55a91697cf00 2024-04-26T18:27:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641af420 con 0x55a91697d820 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6ebe4740 10 client.0 _ll_register_callbacks cb 0x7f2a5c013790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a6a465640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9169880e0 0x55a9169fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:57988/0 (socket says 172.21.15.37:57988) 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 -- 172.21.15.37:0/625727161 learned_addr learned my addr 172.21.15.37:0/625727161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69463640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a91699b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 -- 172.21.15.37:0/625727161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 msgr2=0x55a91699b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a91699b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 -- 172.21.15.37:0/625727161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9169880e0 msgr2=0x55a9169fd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9169880e0 0x55a9169fd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 -- 172.21.15.37:0/625727161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9169fd790 con 0x55a91697cf00 2024-04-26T18:27:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a69c64640 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce890 tx=0x7f2a60004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba4b941a86d72cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b6040 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.714+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f2a600a3ee0 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6009d1e0 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a600a5750 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 10 client.25304 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a9169fd790 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6009d1e0 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a600af030 con 0x55a91697cf00 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 10 client.25304 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 10 client.25304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 20 client.25304 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9169fede0 0x55a916a01200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 4 client.25304 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9168eaee0 con 0x55a9169fede0 2024-04-26T18:27:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.715+0000 7f2a69463640 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9169fede0 0x55a916a01200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.716+0000 7f2a4d7fa640 10 client.25304 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.716+0000 7f2a69463640 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9169fede0 0x55a916a01200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48b40789241a66d server_cookie=c8fab87664e15466 in_seq=0 out_seq=0 2024-04-26T18:27:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a4d7fa640 1 -- 172.21.15.37:0/625727161 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9168eaee0 con 0x55a9169fede0 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a4d7fa640 10 client.25304 handle_command_reply: tid=0 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 client.25304 shutdown 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 msgr2=0x55a91699b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce890 tx=0x7f2a60004a90 comp rx=0 tx=0).stop 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 shutdown_connections 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55a9169fede0 0x55a916a01200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9169880e0 0x55a9169fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a91697d820 0x55a91699b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 --2- 172.21.15.37:0/625727161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a91697cf00 0x55a91699b210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 >> 172.21.15.37:0/625727161 conn(0x55a916979410 msgr2=0x55a9169785b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a4d7fa640 0 client.25304 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 shutdown_connections 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 1 -- 172.21.15.37:0/625727161 wait complete. 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.729+0000 7f2a6ebe4740 20 client.25304 trim_cache size 0 max 16384 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:27:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 msgr2=0x7f2a500a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 0x7f2a500a62b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0099e0 tx=0x7f2a5c004ab0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 msgr2=0x7f2a64074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9f0 tx=0x7f2a6000edf0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 shutdown_connections 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2a500a3df0 0x7f2a500a62b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a64108940 0x7f2a640756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a641032f0 0x7f2a64075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 --2- 172.21.15.37:0/4145109534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a64102930 0x7f2a64074c30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 >> 172.21.15.37:0/4145109534 conn(0x7f2a640fe400 msgr2=0x7f2a640feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 shutdown_connections 2024-04-26T18:27:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:26.730+0000 7f2a6bec8640 1 -- 172.21.15.37:0/4145109534 wait complete. 2024-04-26T18:27:27.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:27 smithi037 ceph-mon[29171]: pgmap v96: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 658 op/s 2024-04-26T18:27:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:27 smithi120 ceph-mon[32244]: pgmap v96: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 658 op/s 2024-04-26T18:27:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:27 smithi170 ceph-mon[32697]: pgmap v96: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 658 op/s 2024-04-26T18:27:28.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:28 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:28.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:28 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:28.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:28 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:29.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:29 smithi037 ceph-mon[29171]: pgmap v97: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:29.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:29 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:29 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:29 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:27:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:29 smithi120 ceph-mon[32244]: pgmap v97: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:29 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:29 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:29.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:29 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:27:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:29 smithi170 ceph-mon[32697]: pgmap v97: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:29 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:29.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:29 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:29.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:29 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:27:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:31 smithi037 ceph-mon[29171]: pgmap v98: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:31 smithi120 ceph-mon[32244]: pgmap v98: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:31 smithi170 ceph-mon[32697]: pgmap v98: 97 pgs: 97 active+clean; 62 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 537 op/s 2024-04-26T18:27:33.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:33 smithi037 ceph-mon[29171]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 858 op/s 2024-04-26T18:27:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:33 smithi120 ceph-mon[32244]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 858 op/s 2024-04-26T18:27:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:33 smithi170 ceph-mon[32697]: pgmap v99: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.4 MiB/s wr, 858 op/s 2024-04-26T18:27:35.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:35 smithi120 ceph-mon[32244]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 688 op/s 2024-04-26T18:27:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:35 smithi170 ceph-mon[32697]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 688 op/s 2024-04-26T18:27:35.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:35 smithi037 ceph-mon[29171]: pgmap v100: 97 pgs: 97 active+clean; 57 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 688 op/s 2024-04-26T18:27:36.401 INFO:tasks.workunit.client.0.smithi037.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-26T18:27:36.700 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xffcdd 2024-04-26T18:27:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:37 smithi120 ceph-mon[32244]: pgmap v101: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 980 op/s 2024-04-26T18:27:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:37 smithi170 ceph-mon[32697]: pgmap v101: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 980 op/s 2024-04-26T18:27:37.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:37 smithi037 ceph-mon[29171]: pgmap v101: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.1 MiB/s wr, 980 op/s 2024-04-26T18:27:38.122 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xffd97 2024-04-26T18:27:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:39 smithi120 ceph-mon[32244]: pgmap v102: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:39 smithi170 ceph-mon[32697]: pgmap v102: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:39.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:39 smithi037 ceph-mon[29171]: pgmap v102: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 -- 172.21.15.37:0/1894166239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 msgr2=0x7ff66010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 --2- 172.21.15.37:0/1894166239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66010f3a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff658009ca0 tx=0x7ff65802a9e0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 -- 172.21.15.37:0/1894166239 shutdown_connections 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 --2- 172.21.15.37:0/1894166239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66010f3a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 --2- 172.21.15.37:0/1894166239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 0x7ff66010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 --2- 172.21.15.37:0/1894166239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 0x7ff660103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 -- 172.21.15.37:0/1894166239 >> 172.21.15.37:0/1894166239 conn(0x7ff6600789b0 msgr2=0x7ff660078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 -- 172.21.15.37:0/1894166239 shutdown_connections 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.830+0000 7ff6671b4640 1 -- 172.21.15.37:0/1894166239 wait complete. 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 Processor -- start 2024-04-26T18:27:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 -- start start 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 0x7ff660199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 0x7ff6601995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112bb0 con 0x7ff66010cf00 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112a30 con 0x7ff660103bc0 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff6671b4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff660112d30 con 0x7ff6601044e0 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff665751640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 0x7ff6601995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff665751640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56978/0 (socket says 172.21.15.37:56978) 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff665751640 1 -- 172.21.15.37:0/3291067447 learned_addr learned my addr 172.21.15.37:0/3291067447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.831+0000 7ff664f50640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 0x7ff660199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 -- 172.21.15.37:0/3291067447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 msgr2=0x7ff6601995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 0x7ff6601995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 -- 172.21.15.37:0/3291067447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 msgr2=0x7ff660199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 0x7ff660199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 -- 172.21.15.37:0/3291067447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66019e080 con 0x7ff66010cf00 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff665751640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff658006010 tx=0x7ff65802fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff65dffb640 1 -- 172.21.15.37:0/3291067447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658033040 con 0x7ff66010cf00 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66019e310 con 0x7ff66010cf00 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff65dffb640 1 -- 172.21.15.37:0/3291067447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65802e7a0 con 0x7ff66010cf00 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff65dffb640 1 -- 172.21.15.37:0/3291067447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658043720 con 0x7ff66010cf00 2024-04-26T18:27:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.832+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601a5c00 con 0x7ff66010cf00 2024-04-26T18:27:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff65dffb640 1 -- 172.21.15.37:0/3291067447 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff658036070 con 0x7ff66010cf00 2024-04-26T18:27:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 Processor -- start 2024-04-26T18:27:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 -- start start 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c80c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c8194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56444c772460 con 0x56444c80bf00 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56444c72c590 con 0x56444c8170e0 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.835+0000 7ff669fab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56444c76e420 con 0x56444c80c820 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c80c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:55470/0 (socket says 172.21.15.37:55470) 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c80c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56980/0 (socket says 172.21.15.37:56980) 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 -- 172.21.15.37:0/2892831567 learned_addr learned my addr 172.21.15.37:0/2892831567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff65ffff640 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c8194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff65dffb640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 0x7ff6500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 -- 172.21.15.37:0/2892831567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 msgr2=0x56444c8194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c8194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 -- 172.21.15.37:0/2892831567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 msgr2=0x56444c80c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c80c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 -- 172.21.15.37:0/2892831567 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444c7390d0 con 0x56444c80c820 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff665751640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 0x7ff6500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff65dffb640 1 -- 172.21.15.37:0/3291067447 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff6580f14a0 con 0x7ff66010cf00 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff664f50640 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff654009cf0 tx=0x7ff654011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4680ebc05b6d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff64affd640 1 -- 172.21.15.37:0/2892831567 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654004230 con 0x56444c80c820 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff64affd640 1 -- 172.21.15.37:0/2892831567 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6540043d0 con 0x56444c80c820 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.836+0000 7ff64affd640 1 -- 172.21.15.37:0/2892831567 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654019880 con 0x56444c80c820 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 -- 172.21.15.37:0/2892831567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 msgr2=0x56444c816ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff665751640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 0x7ff6500a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6580eec90 tx=0x7ff6580b2970 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff654009cf0 tx=0x7ff654011810 comp rx=0 tx=0).stop 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 -- 172.21.15.37:0/2892831567 shutdown_connections 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c8194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80c820 0x56444c816ba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 --2- 172.21.15.37:0/2892831567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c80c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 -- 172.21.15.37:0/2892831567 >> 172.21.15.37:0/2892831567 conn(0x56444c808410 msgr2=0x56444c8087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 -- 172.21.15.37:0/2892831567 shutdown_connections 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.837+0000 7ff669fab740 1 -- 172.21.15.37:0/2892831567 wait complete. 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 -- start start 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 Processor -- start 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c8140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c814b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56444c819d10 con 0x56444c80c820 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56444c6f9b60 con 0x56444c8170e0 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56444c819e90 con 0x56444c80bf00 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff669fab740 10 client.0 _ll_register_callbacks cb 0x56444c71e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff664f50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff664f50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56986/0 (socket says 172.21.15.37:56986) 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff664f50640 1 -- 172.21.15.37:0/4138929650 learned_addr learned my addr 172.21.15.37:0/4138929650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff65ffff640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c8140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.838+0000 7ff65ffff640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c814b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 -- 172.21.15.37:0/4138929650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 msgr2=0x56444c8140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c8140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 -- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 msgr2=0x56444c814b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c814b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 -- 172.21.15.37:0/4138929650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56444c484210 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff65ffff640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c8140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff664f50640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff654003820 tx=0x7ff6540096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a7c3281d5d2c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654027070 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6540043d0 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff654005240 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff654033050 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff669fab740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.839+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56444c550620 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6540043d0 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff654004b70 con 0x56444c80c820 2024-04-26T18:27:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 20 client.15690 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c8271d0 0x56444c8295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 4 client.15690 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56444c780070 con 0x56444c8271d0 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff65ffff640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c8271d0 0x56444c8295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff63ffff640 10 client.15690 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.840+0000 7ff65ffff640 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c8271d0 0x56444c8295c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e358e0eeb2cecbc7 server_cookie=189e4ae59e15b364 in_seq=0 out_seq=0 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff63ffff640 1 -- 172.21.15.37:0/4138929650 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56444c780070 con 0x56444c8271d0 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff63ffff640 10 client.15690 handle_command_reply: tid=0 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 client.15690 shutdown 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 msgr2=0x56444c814610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff654003820 tx=0x7ff6540096f0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 shutdown_connections 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c8271d0 0x56444c8295c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c814b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80c820 0x56444c814610 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 --2- 172.21.15.37:0/4138929650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c8140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 >> 172.21.15.37:0/4138929650 conn(0x56444c808410 msgr2=0x56444c80e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.841+0000 7ff63ffff640 0 client.15690 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.842+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 shutdown_connections 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.842+0000 7ff669fab740 1 -- 172.21.15.37:0/4138929650 wait complete. 2024-04-26T18:27:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.842+0000 7ff669fab740 20 client.15690 trim_cache size 0 max 16384 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.859+0000 7ff669fab740 1 Processor -- start 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.859+0000 7ff669fab740 1 -- start start 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c822bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c80e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601b0f00 con 0x56444c816010 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ae7d0 con 0x56444c8170e0 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff669fab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ac620 con 0x56444c80bf00 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff664f50640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c80e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56988/0 (socket says 172.21.15.37:56988) 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 -- 172.21.15.37:0/1473138426 learned_addr learned my addr 172.21.15.37:0/1473138426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c822bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 -- 172.21.15.37:0/1473138426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 msgr2=0x56444c822bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c822bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 -- 172.21.15.37:0/1473138426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 msgr2=0x56444c80e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c80e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 -- 172.21.15.37:0/1473138426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444c812620 con 0x56444c816010 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff65ffff640 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x56444c8095a0 tx=0x7ff64c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e9947410a5764 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.860+0000 7ff64a7fc640 1 -- 172.21.15.37:0/1473138426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c013c90 con 0x56444c816010 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff64a7fc640 1 -- 172.21.15.37:0/1473138426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff64c013e30 con 0x56444c816010 2024-04-26T18:27:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff64a7fc640 1 -- 172.21.15.37:0/1473138426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c010690 con 0x56444c816010 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 -- 172.21.15.37:0/1473138426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 msgr2=0x56444c82edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x56444c8095a0 tx=0x7ff64c00b740 comp rx=0 tx=0).stop 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 -- 172.21.15.37:0/1473138426 shutdown_connections 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c80e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c816010 0x56444c82edf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 --2- 172.21.15.37:0/1473138426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c80bf00 0x56444c822bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 -- 172.21.15.37:0/1473138426 >> 172.21.15.37:0/1473138426 conn(0x56444c808410 msgr2=0x56444c815900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 -- 172.21.15.37:0/1473138426 shutdown_connections 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.861+0000 7ff669fab740 1 -- 172.21.15.37:0/1473138426 wait complete. 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 -- start start 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 Processor -- start 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 0x56444c81f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c81c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601acc90 con 0x56444c80bf00 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ae100 con 0x56444c8170e0 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ad360 con 0x56444c816010 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff669fab740 10 client.0 _ll_register_callbacks cb 0x7ff64c00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56990/0 (socket says 172.21.15.37:56990) 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff65ffff640 1 -- 172.21.15.37:0/3611067372 learned_addr learned my addr 172.21.15.37:0/3611067372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff664f50640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 0x56444c81f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.862+0000 7ff65ffff640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c81c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 -- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 msgr2=0x56444c81c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c81c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 -- 172.21.15.37:0/3611067372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 msgr2=0x56444c81f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 -- 172.21.15.37:0/3611067372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56444c81c620 con 0x56444c816010 2024-04-26T18:27:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff65ffff640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff664f50640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 0x56444c81f620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56444c815160 tx=0x7ff6540044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e2243349e6e7b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff63ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654027070 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff654019c00 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64c013e30 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff654009ea0 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff669fab740 10 client.25279 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.863+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56444c88dee0 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff64c013e30 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff654004640 con 0x56444c816010 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 10 client.25279 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 10 client.25279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 20 client.25279 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:39.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c88e670 0x56444c890a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 4 client.25279 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56444c779ee0 con 0x56444c88e670 2024-04-26T18:27:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff65ffff640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c88e670 0x56444c890a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff65ffff640 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c88e670 0x56444c890a90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d9aaef6767b7b0 server_cookie=45dfe71ac5fcab6e in_seq=0 out_seq=0 2024-04-26T18:27:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.864+0000 7ff63ffff640 10 client.25279 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff63ffff640 1 -- 172.21.15.37:0/3611067372 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56444c779ee0 con 0x56444c88e670 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff63ffff640 10 client.25279 handle_command_reply: tid=0 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 client.25279 shutdown 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 msgr2=0x56444c81f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 0x56444c81f620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x56444c815160 tx=0x7ff6540044f0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 shutdown_connections 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x56444c88e670 0x56444c890a90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56444c8170e0 0x56444c81c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56444c816010 0x56444c81f620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 --2- 172.21.15.37:0/3611067372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56444c80bf00 0x56444c81f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff63ffff640 0 client.25279 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 >> 172.21.15.37:0/3611067372 conn(0x56444c808410 msgr2=0x56444c814070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.878+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 shutdown_connections 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff669fab740 1 -- 172.21.15.37:0/3611067372 wait complete. 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff669fab740 20 client.25279 trim_cache size 0 max 16384 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:27:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:27:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 msgr2=0x7ff6500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 0x7ff6500a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6580eec90 tx=0x7ff6580b2970 comp rx=0 tx=0).stop 2024-04-26T18:27:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 msgr2=0x7ff66019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff658006010 tx=0x7ff65802fbe0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 shutdown_connections 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff6500a3dc0 0x7ff6500a6280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff66010cf00 0x7ff66019d950 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6601044e0 0x7ff6601995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 --2- 172.21.15.37:0/3291067447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff660103bc0 0x7ff660199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 >> 172.21.15.37:0/3291067447 conn(0x7ff6600789b0 msgr2=0x7ff6600fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 shutdown_connections 2024-04-26T18:27:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:39.879+0000 7ff6671b4640 1 -- 172.21.15.37:0/3291067447 wait complete. 2024-04-26T18:27:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:41 smithi120 ceph-mon[32244]: pgmap v103: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:41 smithi170 ceph-mon[32697]: pgmap v103: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:41.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:41 smithi037 ceph-mon[29171]: pgmap v103: 97 pgs: 97 active+clean; 60 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 612 op/s 2024-04-26T18:27:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:43 smithi120 ceph-mon[32244]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 891 op/s 2024-04-26T18:27:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:43 smithi170 ceph-mon[32697]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 891 op/s 2024-04-26T18:27:43.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:43 smithi037 ceph-mon[29171]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.6 MiB/s wr, 891 op/s 2024-04-26T18:27:45.070 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0xffdb8 2024-04-26T18:27:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:45 smithi120 ceph-mon[32244]: pgmap v105: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 570 op/s 2024-04-26T18:27:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:45 smithi170 ceph-mon[32697]: pgmap v105: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 570 op/s 2024-04-26T18:27:45.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:45 smithi037 ceph-mon[29171]: pgmap v105: 97 pgs: 97 active+clean; 55 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 570 op/s 2024-04-26T18:27:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:47 smithi120 ceph-mon[32244]: pgmap v106: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 827 op/s 2024-04-26T18:27:47.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:47 smithi037 ceph-mon[29171]: pgmap v106: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 827 op/s 2024-04-26T18:27:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:47 smithi170 ceph-mon[32697]: pgmap v106: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 827 op/s 2024-04-26T18:27:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:49 smithi120 ceph-mon[32244]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:49 smithi170 ceph-mon[32697]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:49.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:49 smithi037 ceph-mon[29171]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:51 smithi120 ceph-mon[32244]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:51 smithi170 ceph-mon[32697]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:51.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:51 smithi037 ceph-mon[29171]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 536 op/s 2024-04-26T18:27:52.514 INFO:tasks.workunit.client.0.smithi037.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-26T18:27:52.515 INFO:tasks.workunit.client.0.smithi037.stdout:All 50000 operations completed A-OK! 2024-04-26T18:27:52.515 INFO:tasks.workunit.client.0.smithi037.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-26T18:27:52.515 INFO:tasks.workunit.client.0.smithi037.stdout:Seed set to 1 2024-04-26T18:27:52.642 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T18:27:52.643 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T18:27:52.644 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T18:27:52.644 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T18:27:52.644 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T18:27:52.647 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T18:27:52.649 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero length punch hole 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero length punch hole 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero length punch hole 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero size read 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero length punch hole 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero size read 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero length punch hole 2024-04-26T18:27:52.650 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x4310e9 2024-04-26T18:27:52.653 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x954a45 2024-04-26T18:27:52.684 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x9bc582 2024-04-26T18:27:52.704 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x9eecbc 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 -- 172.21.15.37:0/142245646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc101a70 msgr2=0x7f10dc1075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 --2- 172.21.15.37:0/142245646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc101a70 0x7f10dc1075a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009900 tx=0x7f10d002a410 comp rx=0 tx=0).stop 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 -- 172.21.15.37:0/142245646 shutdown_connections 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 --2- 172.21.15.37:0/142245646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 --2- 172.21.15.37:0/142245646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc101a70 0x7f10dc1075a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 --2- 172.21.15.37:0/142245646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc1010c0 0x7f10dc1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 -- 172.21.15.37:0/142245646 >> 172.21.15.37:0/142245646 conn(0x7f10dc077dc0 msgr2=0x7f10dc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 -- 172.21.15.37:0/142245646 shutdown_connections 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.980+0000 7f10e14c4640 1 -- 172.21.15.37:0/142245646 wait complete. 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 Processor -- start 2024-04-26T18:27:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 -- start start 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc1010c0 0x7f10dc102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112bb0 con 0x7f10dc1010c0 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112a30 con 0x7f10dc102ab0 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10e14c4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc112d30 con 0x7f10dc107c80 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10daffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc1010c0 0x7f10dc102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:58610/0 (socket says 172.21.15.37:58610) 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10db7fe640 1 -- 172.21.15.37:0/3432100063 learned_addr learned my addr 172.21.15.37:0/3432100063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.981+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:47472/0 (socket says 172.21.15.37:47472) 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 -- 172.21.15.37:0/3432100063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 msgr2=0x7f10dc102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 -- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc1010c0 msgr2=0x7f10dc102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc1010c0 0x7f10dc102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 -- 172.21.15.37:0/3432100063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a7940 con 0x7f10dc102ab0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10da7fc640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10db7fe640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10d400eb10 tx=0x7f10d400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10c3fff640 1 -- 172.21.15.37:0/3432100063 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004b20 con 0x7f10dc102ab0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10c3fff640 1 -- 172.21.15.37:0/3432100063 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d4004cc0 con 0x7f10dc102ab0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a7c30 con 0x7f10dc102ab0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10c3fff640 1 -- 172.21.15.37:0/3432100063 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4010670 con 0x7f10dc102ab0 2024-04-26T18:27:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.982+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a8100 con 0x7f10dc102ab0 2024-04-26T18:27:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.984+0000 7f10c3fff640 1 -- 172.21.15.37:0/3432100063 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10d4005000 con 0x7f10dc102ab0 2024-04-26T18:27:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 Processor -- start 2024-04-26T18:27:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 -- start start 2024-04-26T18:27:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561af42d8460 con 0x561af4371f00 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561af4292590 con 0x561af437d0e0 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10e41d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561af42d4420 con 0x561af4372820 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36024/0 (socket says 172.21.15.37:36024) 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10daffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 -- 172.21.15.37:0/1389485545 learned_addr learned my addr 172.21.15.37:0/1389485545 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10c3fff640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 0x7f10c40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 -- 172.21.15.37:0/1389485545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 msgr2=0x561af437cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10db7fe640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 0x7f10c40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 -- 172.21.15.37:0/1389485545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 msgr2=0x561af437f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10da7fc640 1 -- 172.21.15.37:0/1389485545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af429f0d0 con 0x561af4371f00 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.985+0000 7f10c3fff640 1 -- 172.21.15.37:0/3432100063 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f10d40d0cd0 con 0x7f10dc102ab0 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10daffd640 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10da7fc640 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f10d0005c40 tx=0x7f10d0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc2b603516cc9be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10c17fa640 1 -- 172.21.15.37:0/1389485545 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0044020 con 0x561af4371f00 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10db7fe640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 0x7f10c40a62b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10d40cea10 tx=0x7f10d400cf70 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10c17fa640 1 -- 172.21.15.37:0/1389485545 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0005040 con 0x561af4371f00 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.986+0000 7f10c17fa640 1 -- 172.21.15.37:0/1389485545 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0035040 con 0x561af4371f00 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- 172.21.15.37:0/1389485545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 msgr2=0x561af43722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f10d0005c40 tx=0x7f10d0005b50 comp rx=0 tx=0).stop 2024-04-26T18:27:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- 172.21.15.37:0/1389485545 shutdown_connections 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 --2- 172.21.15.37:0/1389485545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4371f00 0x561af43722e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- 172.21.15.37:0/1389485545 >> 172.21.15.37:0/1389485545 conn(0x561af436e410 msgr2=0x561af436e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- 172.21.15.37:0/1389485545 shutdown_connections 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- 172.21.15.37:0/1389485545 wait complete. 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.987+0000 7f10e41d8740 1 -- start start 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 Processor -- start 2024-04-26T18:27:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af43918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561af437fcb0 con 0x561af437ad20 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561af425fb60 con 0x561af437d0e0 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561af437fe30 con 0x561af4372820 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10e41d8740 10 client.0 _ll_register_callbacks cb 0x561af42842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af43918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10daffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:58630/0 (socket says 172.21.15.37:58630) 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.988+0000 7f10daffd640 1 -- 172.21.15.37:0/342439929 learned_addr learned my addr 172.21.15.37:0/342439929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 -- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 msgr2=0x561af437a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 -- 172.21.15.37:0/342439929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 msgr2=0x561af43918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af43918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 -- 172.21.15.37:0/342439929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af3fea210 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10daffd640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10da7fc640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437a2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10d003e000 tx=0x7f10d0005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82bf88b24e52eb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10beff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0050070 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f10d0004b80 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d0005040 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10d0059050 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10e41d8740 10 client.25291 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561af40b6620 con 0x561af4372820 2024-04-26T18:27:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.989+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0005040 con 0x561af4372820 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f10d0053020 con 0x561af4372820 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 10 client.25291 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 20 client.25291 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af4395e70 0x561af4398260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 4 client.25291 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af42e6070 con 0x561af4395e70 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10daffd640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af4395e70 0x561af4398260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10daffd640 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af4395e70 0x561af4398260 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6dcf8d9cf39f3b8 server_cookie=43dbef5d2911977 in_seq=0 out_seq=0 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.990+0000 7f10beff5640 10 client.25291 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10beff5640 1 -- 172.21.15.37:0/342439929 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561af42e6070 con 0x561af4395e70 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10beff5640 10 client.25291 handle_command_reply: tid=0 2024-04-26T18:27:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 client.25291 shutdown 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 msgr2=0x561af437a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437a2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10d003e000 tx=0x7f10d0005e50 comp rx=0 tx=0).stop 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 shutdown_connections 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af4395e70 0x561af4398260 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af437a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10beff5640 0 client.25291 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af43918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 --2- 172.21.15.37:0/342439929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af4372820 0x561af437a2a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 >> 172.21.15.37:0/342439929 conn(0x561af436e410 msgr2=0x561af4374190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 shutdown_connections 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.991+0000 7f10e41d8740 1 -- 172.21.15.37:0/342439929 wait complete. 2024-04-26T18:27:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:52.992+0000 7f10e41d8740 20 client.25291 trim_cache size 0 max 16384 2024-04-26T18:27:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 Processor -- start 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 -- start start 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 0x561af43918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 0x561af439b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af439b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1aec10 con 0x561af4372820 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1ac4e0 con 0x561af437d0e0 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.008+0000 7f10e41d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1aa330 con 0x561af437ad20 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 0x561af43918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 0x561af439b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 0x561af43918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36030/0 (socket says 172.21.15.37:36030) 2024-04-26T18:27:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10da7fc640 1 -- 172.21.15.37:0/2124454596 learned_addr learned my addr 172.21.15.37:0/2124454596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10da7fc640 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af439b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 -- 172.21.15.37:0/2124454596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 msgr2=0x561af439b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af439b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 -- 172.21.15.37:0/2124454596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 msgr2=0x561af43918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 0x561af43918a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 -- 172.21.15.37:0/2124454596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af437b200 con 0x561af437ad20 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10daffd640 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 0x561af439b280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x561af425fc00 tx=0x7f10cc00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c49520bf22e9c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10c0ff9640 1 -- 172.21.15.37:0/2124454596 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004020 con 0x561af437ad20 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10c0ff9640 1 -- 172.21.15.37:0/2124454596 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc024030 con 0x561af437ad20 2024-04-26T18:27:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.009+0000 7f10c0ff9640 1 -- 172.21.15.37:0/2124454596 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc010b90 con 0x561af437ad20 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 -- 172.21.15.37:0/2124454596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 msgr2=0x561af439b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 0x561af439b280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x561af425fc00 tx=0x7f10cc00b410 comp rx=0 tx=0).stop 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 -- 172.21.15.37:0/2124454596 shutdown_connections 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af437d0e0 0x561af439b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437ad20 0x561af439b280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 --2- 172.21.15.37:0/2124454596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af4372820 0x561af43918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 -- 172.21.15.37:0/2124454596 >> 172.21.15.37:0/2124454596 conn(0x561af436e410 msgr2=0x561af43754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 -- 172.21.15.37:0/2124454596 shutdown_connections 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.010+0000 7f10e41d8740 1 -- 172.21.15.37:0/2124454596 wait complete. 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 -- start start 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 Processor -- start 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 0x561af4398f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af4399470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437d0e0 0x561af43999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1aa9a0 con 0x561af437ad20 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1abe10 con 0x561af4372820 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1ab070 con 0x561af437d0e0 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10e41d8740 10 client.0 _ll_register_callbacks cb 0x7f10cc00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10daffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af4399470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10daffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af4399470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36042/0 (socket says 172.21.15.37:36042) 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10daffd640 1 -- 172.21.15.37:0/4269387554 learned_addr learned my addr 172.21.15.37:0/4269387554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:27:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 0x561af4398f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437d0e0 0x561af43999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 -- 172.21.15.37:0/4269387554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437d0e0 msgr2=0x561af43999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437d0e0 0x561af43999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 -- 172.21.15.37:0/4269387554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 msgr2=0x561af4399470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af4399470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.011+0000 7f10da7fc640 1 -- 172.21.15.37:0/4269387554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af43fb2a0 con 0x561af4372820 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10da7fc640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 0x561af4398f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10d0031620 tx=0x7f10d0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf56ece2ca81eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10beff5640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0050070 con 0x561af4372820 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f10d004c800 con 0x561af4372820 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10cc024030 con 0x561af4372820 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10d0035040 con 0x561af4372820 2024-04-26T18:27:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10e41d8740 10 client.25340 fetch_fsmap learned FSMap version 27 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.012+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561af43fc570 con 0x561af4372820 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc024030 con 0x561af4372820 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f10d0053040 con 0x561af4372820 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 10 client.25340 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 10 client.25340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 20 client.25340 populate_metadata read hostname 'smithi037' 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af43fcc30 0x561af43ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 4 client.25340 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561af42dfee0 con 0x561af43fcc30 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10daffd640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af43fcc30 0x561af43ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10beff5640 10 client.25340 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.013+0000 7f10daffd640 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af43fcc30 0x561af43ff050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea87092ae4d3b99 server_cookie=72354612c643b7e3 in_seq=0 out_seq=0 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10beff5640 1 -- 172.21.15.37:0/4269387554 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561af42dfee0 con 0x561af43fcc30 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10beff5640 10 client.25340 handle_command_reply: tid=0 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 client.25340 shutdown 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 msgr2=0x561af4398f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 0x561af4398f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10d0031620 tx=0x7f10d0005fd0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 shutdown_connections 2024-04-26T18:27:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x561af43fcc30 0x561af43ff050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561af437d0e0 0x561af43999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10beff5640 0 client.25340 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561af437ad20 0x561af4399470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 --2- 172.21.15.37:0/4269387554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561af4372820 0x561af4398f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 >> 172.21.15.37:0/4269387554 conn(0x561af436e410 msgr2=0x561af436cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 shutdown_connections 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 1 -- 172.21.15.37:0/4269387554 wait complete. 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.031+0000 7f10e41d8740 20 client.25340 trim_cache size 0 max 16384 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:27:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:27:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:27:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 msgr2=0x7f10c40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 0x7f10c40a62b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10d40cea10 tx=0x7f10d400cf70 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 msgr2=0x7f10dc1a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10d400eb10 tx=0x7f10d400c2d0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 shutdown_connections 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f10c40a3df0 0x7f10c40a62b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc102ab0 0x7f10dc1a7240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f10dc107c80 0x7f10dc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 --2- 172.21.15.37:0/3432100063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10dc1010c0 0x7f10dc102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.032+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 >> 172.21.15.37:0/3432100063 conn(0x7f10dc077dc0 msgr2=0x7f10dc105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.034+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 shutdown_connections 2024-04-26T18:27:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:27:53.034+0000 7f10e14c4640 1 -- 172.21.15.37:0/3432100063 wait complete. 2024-04-26T18:27:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:53 smithi170 ceph-mon[32697]: pgmap v109: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 769 op/s 2024-04-26T18:27:53.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:53 smithi037 ceph-mon[29171]: pgmap v109: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 769 op/s 2024-04-26T18:27:53.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:53 smithi120 ceph-mon[32244]: pgmap v109: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.9 MiB/s wr, 769 op/s 2024-04-26T18:27:54.732 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x9fdd84 2024-04-26T18:27:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:55 smithi170 ceph-mon[32697]: pgmap v110: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 490 op/s 2024-04-26T18:27:55.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:55 smithi037 ceph-mon[29171]: pgmap v110: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 490 op/s 2024-04-26T18:27:55.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:55 smithi120 ceph-mon[32244]: pgmap v110: 97 pgs: 97 active+clean; 63 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 490 op/s 2024-04-26T18:27:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:57 smithi170 ceph-mon[32697]: pgmap v111: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 788 op/s 2024-04-26T18:27:57.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:57 smithi037 ceph-mon[29171]: pgmap v111: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 788 op/s 2024-04-26T18:27:57.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:57 smithi120 ceph-mon[32244]: pgmap v111: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 788 op/s 2024-04-26T18:27:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:27:59 smithi170 ceph-mon[32697]: pgmap v112: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:27:59.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:27:59 smithi037 ceph-mon[29171]: pgmap v112: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:27:59.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:27:59 smithi120 ceph-mon[32244]: pgmap v112: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:28:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:01 smithi170 ceph-mon[32697]: pgmap v113: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:28:01.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:01 smithi037 ceph-mon[29171]: pgmap v113: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:28:01.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:01 smithi120 ceph-mon[32244]: pgmap v113: 97 pgs: 97 active+clean; 59 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-04-26T18:28:02.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:02 smithi170 ceph-mon[32697]: pgmap v114: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.9 MiB/s wr, 858 op/s 2024-04-26T18:28:02.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:02 smithi037 ceph-mon[29171]: pgmap v114: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.9 MiB/s wr, 858 op/s 2024-04-26T18:28:02.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:02 smithi120 ceph-mon[32244]: pgmap v114: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.9 MiB/s wr, 858 op/s 2024-04-26T18:28:04.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:04 smithi037 ceph-mon[29171]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 624 op/s 2024-04-26T18:28:05.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:04 smithi120 ceph-mon[32244]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 624 op/s 2024-04-26T18:28:05.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:04 smithi170 ceph-mon[32697]: pgmap v115: 97 pgs: 97 active+clean; 68 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 624 op/s 2024-04-26T18:28:06.100 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x9ff773 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 -- 172.21.15.37:0/74635025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744107860 msgr2=0x7fa744107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 --2- 172.21.15.37:0/74635025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744107860 0x7fa744107c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa734009930 tx=0x7fa73402a420 comp rx=0 tx=0).stop 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 -- 172.21.15.37:0/74635025 shutdown_connections 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 --2- 172.21.15.37:0/74635025 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa74410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 --2- 172.21.15.37:0/74635025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744101850 0x7fa744101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 --2- 172.21.15.37:0/74635025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744107860 0x7fa744107c40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 -- 172.21.15.37:0/74635025 >> 172.21.15.37:0/74635025 conn(0x7fa744077bb0 msgr2=0x7fa744077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 -- 172.21.15.37:0/74635025 shutdown_connections 2024-04-26T18:28:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.131+0000 7fa7497da640 1 -- 172.21.15.37:0/74635025 wait complete. 2024-04-26T18:28:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 Processor -- start 2024-04-26T18:28:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 -- start start 2024-04-26T18:28:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa744103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 0x7fa744103ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa744111b20 con 0x7fa744101850 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441119a0 con 0x7fa744102210 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7497da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa744111ca0 con 0x7fa744107860 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37734/0 (socket says 172.21.15.37:37734) 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa742ffd640 1 -- 172.21.15.37:0/1242378190 learned_addr learned my addr 172.21.15.37:0/1242378190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7427fc640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa744103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.132+0000 7fa7437fe640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 0x7fa744103ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 -- 172.21.15.37:0/1242378190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 msgr2=0x7fa744103ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 0x7fa744103ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 -- 172.21.15.37:0/1242378190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 msgr2=0x7fa744103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa744103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 -- 172.21.15.37:0/1242378190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7441a7690 con 0x7fa744101850 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa7427fc640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa744103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa7437fe640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 0x7fa744103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa742ffd640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa734005940 tx=0x7fa7340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa72bfff640 1 -- 172.21.15.37:0/1242378190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73403f030 con 0x7fa744101850 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa72bfff640 1 -- 172.21.15.37:0/1242378190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7340044c0 con 0x7fa744101850 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa72bfff640 1 -- 172.21.15.37:0/1242378190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734035070 con 0x7fa744101850 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7441a7920 con 0x7fa744101850 2024-04-26T18:28:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.133+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7441a7dc0 con 0x7fa744101850 2024-04-26T18:28:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa72bfff640 1 -- 172.21.15.37:0/1242378190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa734042030 con 0x7fa744101850 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 Processor -- start 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 -- start start 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3ecab8460 con 0x55b3ecb52820 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3eca72590 con 0x55b3ecb51f00 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.136+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3ecab4420 con 0x55b3ecb5d0e0 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56680/0 (socket says 172.21.15.37:56680) 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 -- 172.21.15.37:0/337639846 learned_addr learned my addr 172.21.15.37:0/337639846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa72bfff640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 0x7fa72c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7427fc640 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 -- 172.21.15.37:0/337639846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 msgr2=0x55b3ecb5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 -- 172.21.15.37:0/337639846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 msgr2=0x55b3ecb5cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 -- 172.21.15.37:0/337639846 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3eca7f0d0 con 0x55b3ecb51f00 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa72bfff640 1 -- 172.21.15.37:0/1242378190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa7340f0b10 con 0x7fa744101850 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 0x7fa72c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7437fe640 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0118d0 tx=0x7fa73c011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91ada987fa2a8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7297fa640 1 -- 172.21.15.37:0/337639846 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c017050 con 0x55b3ecb51f00 2024-04-26T18:28:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7297fa640 1 -- 172.21.15.37:0/337639846 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa73c00e3b0 con 0x55b3ecb51f00 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.137+0000 7fa7297fa640 1 -- 172.21.15.37:0/337639846 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01a5c0 con 0x55b3ecb51f00 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 -- 172.21.15.37:0/337639846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 msgr2=0x55b3ecb522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa7437fe640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 0x7fa72c0a62b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa73c023000 tx=0x7fa73c024000 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0118d0 tx=0x7fa73c011d90 comp rx=0 tx=0).stop 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 -- 172.21.15.37:0/337639846 shutdown_connections 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/337639846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb522e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 -- 172.21.15.37:0/337639846 >> 172.21.15.37:0/337639846 conn(0x55b3ecb4e410 msgr2=0x55b3ecb4e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 -- 172.21.15.37:0/337639846 shutdown_connections 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.138+0000 7fa74c5c9740 1 -- 172.21.15.37:0/337639846 wait complete. 2024-04-26T18:28:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 -- start start 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 Processor -- start 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb5a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3ecb5fd10 con 0x55b3ecb52820 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b3eca3fb60 con 0x55b3ecb51f00 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3ecb5fe90 con 0x55b3ecb5d0e0 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa74c5c9740 10 client.0 _ll_register_callbacks cb 0x55b3eca642b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb5a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37754/0 (socket says 172.21.15.37:37754) 2024-04-26T18:28:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 -- 172.21.15.37:0/3913281077 learned_addr learned my addr 172.21.15.37:0/3913281077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7437fe640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 -- 172.21.15.37:0/3913281077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 msgr2=0x55b3ecb5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 -- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 msgr2=0x55b3ecb5a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb5a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 -- 172.21.15.37:0/3913281077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ec7ca210 con 0x55b3ecb52820 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb5a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa738012a10 tx=0x7fa738012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f668f7aef13c931 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.139+0000 7fa726ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738004050 con 0x55b3ecb52820 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa7380041f0 con 0x55b3ecb52820 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa73c00e3b0 con 0x55b3ecb52820 2024-04-26T18:28:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa7380043f0 con 0x55b3ecb52820 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3ec896620 con 0x55b3ecb52820 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa73c00e3b0 con 0x55b3ecb52820 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa738003b60 con 0x55b3ecb52820 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 20 client.15726 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecb6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 4 client.15726 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ecac6070 con 0x55b3ecb6d310 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.140+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecb6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.141+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecb6f740 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb0c3713f1cf8fc server_cookie=59e9312ad1929a41 in_seq=0 out_seq=0 2024-04-26T18:28:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.141+0000 7fa726ff5640 10 client.15726 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa726ff5640 1 -- 172.21.15.37:0/3913281077 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3ecac6070 con 0x55b3ecb6d310 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa726ff5640 10 client.15726 handle_command_reply: tid=0 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 client.15726 shutdown 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 msgr2=0x55b3ecb5a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa738012a10 tx=0x7fa738012e10 comp rx=0 tx=0).stop 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 shutdown_connections 2024-04-26T18:28:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecb6f740 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa726ff5640 0 client.15726 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a5c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3913281077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb51f00 0x55b3ecb5a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 >> 172.21.15.37:0/3913281077 conn(0x55b3ecb4e410 msgr2=0x55b3ecb54190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 shutdown_connections 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3913281077 wait complete. 2024-04-26T18:28:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.142+0000 7fa74c5c9740 20 client.15726 trim_cache size 0 max 16384 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 Processor -- start 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 -- start start 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441b2ff0 con 0x55b3ecb52820 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441b08c0 con 0x55b3ecb5d0e0 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441ae710 con 0x55b3ecb51f00 2024-04-26T18:28:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa7427fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:55878/0 (socket says 172.21.15.37:55878) 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa7427fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37756/0 (socket says 172.21.15.37:37756) 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa742ffd640 1 -- 172.21.15.37:0/2107688746 learned_addr learned my addr 172.21.15.37:0/2107688746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.159+0000 7fa7437fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 -- 172.21.15.37:0/2107688746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 msgr2=0x55b3ecb750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 -- 172.21.15.37:0/2107688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 msgr2=0x55b3ecb5a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 -- 172.21.15.37:0/2107688746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ecb4efe0 con 0x55b3ecb51f00 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa7427fc640 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa742ffd640 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa734031440 tx=0x7fa7340f4660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107f1ae44f810620 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa728ff9640 1 -- 172.21.15.37:0/2107688746 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734049040 con 0x55b3ecb51f00 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa728ff9640 1 -- 172.21.15.37:0/2107688746 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7340c5710 con 0x55b3ecb51f00 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.160+0000 7fa728ff9640 1 -- 172.21.15.37:0/2107688746 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7340f5040 con 0x55b3ecb51f00 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- 172.21.15.37:0/2107688746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 msgr2=0x55b3ecb68bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa734031440 tx=0x7fa7340f4660 comp rx=0 tx=0).stop 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- 172.21.15.37:0/2107688746 shutdown_connections 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb52820 0x55b3ecb5a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/2107688746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb51f00 0x55b3ecb68bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- 172.21.15.37:0/2107688746 >> 172.21.15.37:0/2107688746 conn(0x55b3ecb4e410 msgr2=0x55b3ecb5a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- 172.21.15.37:0/2107688746 shutdown_connections 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- 172.21.15.37:0/2107688746 wait complete. 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 -- start start 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.161+0000 7fa74c5c9740 1 Processor -- start 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 0x55b3ecb65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441aed80 con 0x55b3ecb51f00 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441b01f0 con 0x55b3ecb5d0e0 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa7441af450 con 0x55b3ecb52820 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa74c5c9740 10 client.0 _ll_register_callbacks cb 0x7fa738009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37760/0 (socket says 172.21.15.37:37760) 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 -- 172.21.15.37:0/3739541240 learned_addr learned my addr 172.21.15.37:0/3739541240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa7437fe640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 0x55b3ecb65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 -- 172.21.15.37:0/3739541240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 msgr2=0x55b3ecb65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 0x55b3ecb65620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 -- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 msgr2=0x55b3ecb620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 -- 172.21.15.37:0/3739541240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ecb62900 con 0x55b3ecb51f00 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 0x55b3ecb65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa7437fe640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa742ffd640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa7340b9520 tx=0x7fa73402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1aa6f4d72508f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa726ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7340d5020 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.162+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa7340bae50 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7340c5710 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa734049070 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 10 client.15732 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b3ecbd4470 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7340c5710 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fa7340c5280 con 0x55b3ecb51f00 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 20 client.15732 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecbd6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 4 client.15732 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3ecabfee0 con 0x55b3ecb6d310 2024-04-26T18:28:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.163+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecbd6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.164+0000 7fa7427fc640 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecbd6b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1aca1a732a1f2b6 server_cookie=eecfd4cde1fcb4a5 in_seq=0 out_seq=0 2024-04-26T18:28:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.164+0000 7fa726ff5640 10 client.15732 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa726ff5640 1 -- 172.21.15.37:0/3739541240 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3ecabfee0 con 0x55b3ecb6d310 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa726ff5640 10 client.15732 handle_command_reply: tid=0 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 client.15732 shutdown 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 msgr2=0x55b3ecb650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa7340b9520 tx=0x7fa73402a420 comp rx=0 tx=0).stop 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 shutdown_connections 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55b3ecb6d310 0x55b3ecbd6b10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b3ecb5d0e0 0x55b3ecb620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3ecb52820 0x55b3ecb65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 --2- 172.21.15.37:0/3739541240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3ecb51f00 0x55b3ecb650e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa726ff5640 0 client.15732 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 >> 172.21.15.37:0/3739541240 conn(0x55b3ecb4e410 msgr2=0x55b3ecb538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 shutdown_connections 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 1 -- 172.21.15.37:0/3739541240 wait complete. 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.181+0000 7fa74c5c9740 20 client.15732 trim_cache size 0 max 16384 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:28:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:28:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 msgr2=0x7fa72c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 0x7fa72c0a62b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa73c023000 tx=0x7fa73c024000 comp rx=0 tx=0).stop 2024-04-26T18:28:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 msgr2=0x7fa744103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa734005940 tx=0x7fa7340040c0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 shutdown_connections 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa72c0a3df0 0x7fa72c0a62b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa744102210 0x7fa744103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa744107860 0x7fa744103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 --2- 172.21.15.37:0/1242378190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa744101850 0x7fa744103420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 >> 172.21.15.37:0/1242378190 conn(0x7fa744077bb0 msgr2=0x7fa74406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 shutdown_connections 2024-04-26T18:28:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:06.182+0000 7fa7497da640 1 -- 172.21.15.37:0/1242378190 wait complete. 2024-04-26T18:28:06.520 INFO:tasks.workunit.client.0.smithi037.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-26T18:28:06.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:06 smithi037 ceph-mon[29171]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 940 op/s 2024-04-26T18:28:07.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:06 smithi120 ceph-mon[32244]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 940 op/s 2024-04-26T18:28:07.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:06 smithi170 ceph-mon[32697]: pgmap v116: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.2 MiB/s wr, 940 op/s 2024-04-26T18:28:08.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:08 smithi037 ceph-mon[29171]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:09.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:08 smithi120 ceph-mon[32244]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:09.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:08 smithi170 ceph-mon[32697]: pgmap v117: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:10.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:10 smithi037 ceph-mon[29171]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:11.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:10 smithi120 ceph-mon[32244]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:11.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:10 smithi170 ceph-mon[32697]: pgmap v118: 97 pgs: 97 active+clean; 77 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.6 MiB/s wr, 642 op/s 2024-04-26T18:28:12.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:12 smithi037 ceph-mon[29171]: pgmap v119: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 947 op/s 2024-04-26T18:28:13.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:12 smithi120 ceph-mon[32244]: pgmap v119: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 947 op/s 2024-04-26T18:28:13.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:12 smithi170 ceph-mon[32697]: pgmap v119: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.3 MiB/s wr, 947 op/s 2024-04-26T18:28:14.124 INFO:tasks.workunit.client.0.smithi037.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-26T18:28:14.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:14 smithi037 ceph-mon[29171]: pgmap v120: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 620 op/s 2024-04-26T18:28:15.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:14 smithi120 ceph-mon[32244]: pgmap v120: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 620 op/s 2024-04-26T18:28:15.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:14 smithi170 ceph-mon[32697]: pgmap v120: 97 pgs: 97 active+clean; 73 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 620 op/s 2024-04-26T18:28:16.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:16 smithi037 ceph-mon[29171]: pgmap v121: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.2 MiB/s wr, 909 op/s 2024-04-26T18:28:17.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:16 smithi120 ceph-mon[32244]: pgmap v121: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.2 MiB/s wr, 909 op/s 2024-04-26T18:28:17.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:16 smithi170 ceph-mon[32697]: pgmap v121: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.2 MiB/s wr, 909 op/s 2024-04-26T18:28:18.278 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x9ffb01 2024-04-26T18:28:18.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:18 smithi037 ceph-mon[29171]: pgmap v122: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:19.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:18 smithi120 ceph-mon[32244]: pgmap v122: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:19.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:18 smithi170 ceph-mon[32697]: pgmap v122: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 -- 172.21.15.37:0/3398304019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c10f2f0 msgr2=0x7f7e9c1116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 --2- 172.21.15.37:0/3398304019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c1116e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009ca0 tx=0x7f7e9402a950 comp rx=0 tx=0).stop 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 -- 172.21.15.37:0/3398304019 shutdown_connections 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 --2- 172.21.15.37:0/3398304019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c1116e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 --2- 172.21.15.37:0/3398304019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 0x7f7e9c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 --2- 172.21.15.37:0/3398304019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c101740 0x7f7e9c101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 -- 172.21.15.37:0/3398304019 >> 172.21.15.37:0/3398304019 conn(0x7f7e9c078980 msgr2=0x7f7e9c078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 -- 172.21.15.37:0/3398304019 shutdown_connections 2024-04-26T18:28:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.282+0000 7f7ea3989640 1 -- 172.21.15.37:0/3398304019 wait complete. 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 Processor -- start 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 -- start start 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 0x7f7e9c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c114e00 con 0x7f7e9c101740 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c114c80 con 0x7f7e9c10f2f0 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea3989640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c114f80 con 0x7f7e9c102060 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42390/0 (socket says 172.21.15.37:42390) 2024-04-26T18:28:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea0f24640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 0x7f7e9c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.283+0000 7f7ea1725640 1 -- 172.21.15.37:0/2417867846 learned_addr learned my addr 172.21.15.37:0/2417867846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 -- 172.21.15.37:0/2417867846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 msgr2=0x7f7e9c19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 0x7f7e9c19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 -- 172.21.15.37:0/2417867846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c10f2f0 msgr2=0x7f7e9c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 -- 172.21.15.37:0/2417867846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c1a0310 con 0x7f7e9c101740 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea1725640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7e927fc640 1 -- 172.21.15.37:0/2417867846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800cd80 con 0x7f7e9c101740 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7e927fc640 1 -- 172.21.15.37:0/2417867846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e980108e0 con 0x7f7e9c101740 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c1a0600 con 0x7f7e9c101740 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7e927fc640 1 -- 172.21.15.37:0/2417867846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98018980 con 0x7f7e9c101740 2024-04-26T18:28:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.284+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a7e90 con 0x7f7e9c101740 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.287+0000 7f7e927fc640 1 -- 172.21.15.37:0/2417867846 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e98010430 con 0x7f7e9c101740 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 Processor -- start 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 -- start start 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c65bce8460 con 0x55c65bd81f00 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c65bca2590 con 0x55c65bd82820 2024-04-26T18:28:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea677c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c65bce4420 con 0x55c65bd8d0e0 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7e927fc640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 0x7f7e880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd822e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42400/0 (socket says 172.21.15.37:42400) 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.288+0000 7f7ea1f26640 1 -- 172.21.15.37:0/787765951 learned_addr learned my addr 172.21.15.37:0/787765951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7e927fc640 1 -- 172.21.15.37:0/2417867846 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7e980d0a80 con 0x7f7e9c101740 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 0x7f7e880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 -- 172.21.15.37:0/787765951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 msgr2=0x55c65bd8f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 -- 172.21.15.37:0/787765951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 msgr2=0x55c65bd822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 -- 172.21.15.37:0/787765951 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65bcaf0d0 con 0x55c65bd82820 2024-04-26T18:28:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1725640 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8cba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ce1c0 tx=0x7f7e98018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dc672e4e8516ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7e877fe640 1 -- 172.21.15.37:0/787765951 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98026040 con 0x55c65bd82820 2024-04-26T18:28:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.289+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 0x7f7e880a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e94004ad0 tx=0x7f7e9402a950 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.290+0000 7f7e877fe640 1 -- 172.21.15.37:0/787765951 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e980a0dc0 con 0x55c65bd82820 2024-04-26T18:28:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.290+0000 7f7e877fe640 1 -- 172.21.15.37:0/787765951 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98099130 con 0x55c65bd82820 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.290+0000 7f7ea677c740 1 -- 172.21.15.37:0/787765951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 msgr2=0x55c65bd8cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.290+0000 7f7ea677c740 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8cba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e980ce1c0 tx=0x7f7e98018420 comp rx=0 tx=0).stop 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.290+0000 7f7ea677c740 1 -- 172.21.15.37:0/787765951 shutdown_connections 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8cba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- 172.21.15.37:0/787765951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 -- 172.21.15.37:0/787765951 >> 172.21.15.37:0/787765951 conn(0x55c65bd7e410 msgr2=0x55c65bd7e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 -- 172.21.15.37:0/787765951 shutdown_connections 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 -- 172.21.15.37:0/787765951 wait complete. 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 -- start start 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 Processor -- start 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd8a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.291+0000 7f7ea677c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c65bd8fc50 con 0x55c65bd81f00 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea677c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c65bc6fb60 con 0x55c65bd82820 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea677c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c65bd8fdd0 con 0x55c65bd8d0e0 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea677c740 10 client.0 _ll_register_callbacks cb 0x55c65bc942b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea0f24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea0f24640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:53488/0 (socket says 172.21.15.37:53488) 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd8a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea0f24640 1 -- 172.21.15.37:0/1425024069 learned_addr learned my addr 172.21.15.37:0/1425024069 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 -- 172.21.15.37:0/1425024069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 msgr2=0x55c65bd8ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 -- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 msgr2=0x55c65bd8a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 -- 172.21.15.37:0/1425024069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65b9fa210 con 0x55c65bd81f00 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd8a0f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55c65bd82e10 tx=0x7f7e9800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfe66dac62f3e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980a5680 con 0x55c65bd81f00 2024-04-26T18:28:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7e980be3f0 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.292+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e980a0dc0 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e980be650 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7ea677c740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c65bac6620 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e980a0dc0 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e980988a0 con 0x55c65bd81f00 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.293+0000 7f7ea677c740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea677c740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea677c740 20 client.15744 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65bd9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea677c740 4 client.15744 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c65bcf6070 con 0x55c65bd9d080 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65bd9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7e84ff9640 10 client.15744 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.294+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65bd9f470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec03e3663689282 server_cookie=f8cc907e99165168 in_seq=0 out_seq=0 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1425024069 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c65bcf6070 con 0x55c65bd9d080 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7e84ff9640 10 client.15744 handle_command_reply: tid=0 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 client.15744 shutdown 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 msgr2=0x55c65bd8a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd8a0f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55c65bd82e10 tx=0x7f7e9800cf20 comp rx=0 tx=0).stop 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 shutdown_connections 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65bd9f470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd82820 0x55c65bd8a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7e84ff9640 0 client.15744 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd8d0e0 0x55c65bd8ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1425024069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd81f00 0x55c65bd8a0f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 >> 172.21.15.37:0/1425024069 conn(0x55c65bd7e410 msgr2=0x55c65bd84190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 shutdown_connections 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 1 -- 172.21.15.37:0/1425024069 wait complete. 2024-04-26T18:28:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.295+0000 7f7ea677c740 20 client.15744 trim_cache size 0 max 16384 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 Processor -- start 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 -- start start 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd81f00 0x55c65bd98b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd82820 0x55c65bd848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b3090 con 0x55c65bd8d0e0 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b0960 con 0x55c65bd81f00 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.311+0000 7f7ea677c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1ae7b0 con 0x55c65bd82820 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd81f00 0x55c65bd98b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42416/0 (socket says 172.21.15.37:42416) 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 -- 172.21.15.37:0/3131766518 learned_addr learned my addr 172.21.15.37:0/3131766518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd82820 0x55c65bd848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 -- 172.21.15.37:0/3131766518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd82820 msgr2=0x55c65bd848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd82820 0x55c65bd848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 -- 172.21.15.37:0/3131766518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd81f00 msgr2=0x55c65bd98b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd81f00 0x55c65bd98b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 -- 172.21.15.37:0/3131766518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65bd8fd00 con 0x55c65bd8d0e0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c103920 tx=0x7f7e94004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27961ad4c11f2ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7e86ffd640 1 -- 172.21.15.37:0/3131766518 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94043050 con 0x55c65bd8d0e0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7e86ffd640 1 -- 172.21.15.37:0/3131766518 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94044040 con 0x55c65bd8d0e0 2024-04-26T18:28:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.312+0000 7f7e86ffd640 1 -- 172.21.15.37:0/3131766518 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94034040 con 0x55c65bd8d0e0 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 -- 172.21.15.37:0/3131766518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 msgr2=0x55c65bda4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c103920 tx=0x7f7e94004a60 comp rx=0 tx=0).stop 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 -- 172.21.15.37:0/3131766518 shutdown_connections 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd8d0e0 0x55c65bda4a60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd82820 0x55c65bd848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 --2- 172.21.15.37:0/3131766518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd81f00 0x55c65bd98b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 -- 172.21.15.37:0/3131766518 >> 172.21.15.37:0/3131766518 conn(0x55c65bd7e410 msgr2=0x55c65bd8a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 -- 172.21.15.37:0/3131766518 shutdown_connections 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.313+0000 7f7ea677c740 1 -- 172.21.15.37:0/3131766518 wait complete. 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 -- start start 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 Processor -- start 2024-04-26T18:28:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 0x55c65bd86ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1aee20 con 0x55c65bd82820 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1b0290 con 0x55c65bd8d0e0 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1af4f0 con 0x55c65bd81f00 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea677c740 10 client.0 _ll_register_callbacks cb 0x7f7e8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea0f24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea1725640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 0x55c65bd86ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea1f26640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:53508/0 (socket says 172.21.15.37:53508) 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea0f24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42418/0 (socket says 172.21.15.37:42418) 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.314+0000 7f7ea1f26640 1 -- 172.21.15.37:0/1757926515 learned_addr learned my addr 172.21.15.37:0/1757926515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 -- 172.21.15.37:0/1757926515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 msgr2=0x55c65bd86ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 0x55c65bd86ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 -- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 msgr2=0x55c65bd87920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 -- 172.21.15.37:0/1757926515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65be023a0 con 0x55c65bd82820 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea1f26640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 0x55c65bd86ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea0f24640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55c65bd8fee0 tx=0x7f7e8c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb56cdd0329f15a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7e84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01dab0 con 0x55c65bd82820 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f7e8c01dc50 con 0x55c65bd82820 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.315+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e94044040 con 0x55c65bd82820 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c01de50 con 0x55c65bd82820 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 10 client.15750 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c65be03740 con 0x55c65bd82820 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e94044040 con 0x55c65bd82820 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7e8c029670 con 0x55c65bd82820 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 20 client.15750 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65be05f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 4 client.15750 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c65bcefee0 con 0x55c65bd9d080 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.316+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65be05f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.317+0000 7f7ea1725640 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65be05f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88aa6636b3c6175 server_cookie=20d9a0b6dc267d3f in_seq=0 out_seq=0 2024-04-26T18:28:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.317+0000 7f7e84ff9640 10 client.15750 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7e84ff9640 1 -- 172.21.15.37:0/1757926515 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c65bcefee0 con 0x55c65bd9d080 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7e84ff9640 10 client.15750 handle_command_reply: tid=0 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 client.15750 shutdown 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 msgr2=0x55c65bd873e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55c65bd8fee0 tx=0x7f7e8c012b60 comp rx=0 tx=0).stop 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 shutdown_connections 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c65bd9d080 0x55c65be05f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c65bd8d0e0 0x55c65bd87920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c65bd82820 0x55c65bd873e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7e84ff9640 0 client.15750 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 --2- 172.21.15.37:0/1757926515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c65bd81f00 0x55c65bd86ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 >> 172.21.15.37:0/1757926515 conn(0x55c65bd7e410 msgr2=0x55c65bd89ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 shutdown_connections 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 1 -- 172.21.15.37:0/1757926515 wait complete. 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.332+0000 7f7ea677c740 20 client.15750 trim_cache size 0 max 16384 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:28:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:28:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 msgr2=0x7f7e880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 0x7f7e880a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e94004ad0 tx=0x7f7e9402a950 comp rx=0 tx=0).stop 2024-04-26T18:28:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 msgr2=0x7f7e9c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 shutdown_connections 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f7e880a3e40 0x7f7e880a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e9c10f2f0 0x7f7e9c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e9c102060 0x7f7e9c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 --2- 172.21.15.37:0/2417867846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c101740 0x7f7e9c19b320 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 >> 172.21.15.37:0/2417867846 conn(0x7f7e9c078980 msgr2=0x7f7e9c110800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 shutdown_connections 2024-04-26T18:28:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:19.333+0000 7f7ea3989640 1 -- 172.21.15.37:0/2417867846 wait complete. 2024-04-26T18:28:20.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:20 smithi037 ceph-mon[29171]: pgmap v123: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:21.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:20 smithi120 ceph-mon[32244]: pgmap v123: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:21.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:20 smithi170 ceph-mon[32697]: pgmap v123: 97 pgs: 97 active+clean; 78 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.5 MiB/s wr, 593 op/s 2024-04-26T18:28:22.190 INFO:tasks.workunit.client.0.smithi037.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-26T18:28:22.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:22 smithi037 ceph-mon[29171]: pgmap v124: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 872 op/s 2024-04-26T18:28:23.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:22 smithi120 ceph-mon[32244]: pgmap v124: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 872 op/s 2024-04-26T18:28:23.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:22 smithi170 ceph-mon[32697]: pgmap v124: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 872 op/s 2024-04-26T18:28:23.243 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:28:23.243 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:28:23.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:23.521 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:24.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:24 smithi037 ceph-mon[29171]: pgmap v125: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-04-26T18:28:25.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:24 smithi120 ceph-mon[32244]: pgmap v125: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-04-26T18:28:25.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:24 smithi170 ceph-mon[32697]: pgmap v125: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 567 op/s 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 -- 172.21.15.37:0/2951510780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c00103470 msgr2=0x7f1c001038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 --2- 172.21.15.37:0/2951510780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c00103470 0x7f1c001038f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009990 tx=0x7f1bec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 -- 172.21.15.37:0/2951510780 shutdown_connections 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 --2- 172.21.15.37:0/2951510780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103e30 0x7f1c00110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 --2- 172.21.15.37:0/2951510780 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c001094c0 0x7f1c001098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 --2- 172.21.15.37:0/2951510780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c00103470 0x7f1c001038f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 -- 172.21.15.37:0/2951510780 >> 172.21.15.37:0/2951510780 conn(0x7f1c0007a580 msgr2=0x7f1c0007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 -- 172.21.15.37:0/2951510780 shutdown_connections 2024-04-26T18:28:25.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.402+0000 7f1c06d44700 1 -- 172.21.15.37:0/2951510780 wait complete. 2024-04-26T18:28:25.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 Processor -- start 2024-04-26T18:28:25.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 -- start start 2024-04-26T18:28:25.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 0x7f1c001a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c001094c0 0x7f1c0019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001135c0 con 0x7f1c001094c0 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00113440 con 0x7f1c00103e30 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.403+0000 7f1c06d44700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00113740 con 0x7f1c00103470 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:43014/0 (socket says 172.21.15.37:43014) 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 -- 172.21.15.37:0/1277272979 learned_addr learned my addr 172.21.15.37:0/1277272979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1c052e1700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c001094c0 0x7f1c0019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1c04ae0700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 0x7f1c001a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 -- 172.21.15.37:0/1277272979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 msgr2=0x7f1c001a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 0x7f1c001a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 -- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c001094c0 msgr2=0x7f1c0019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c001094c0 0x7f1c0019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 -- 172.21.15.37:0/1277272979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019c220 con 0x7f1c00103e30 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1c04ae0700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 0x7f1c001a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfffff700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0057c0 tx=0x7f1bec009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec026070 con 0x7f1c00103e30 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bec021410 con 0x7f1c00103e30 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.404+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019c4b0 con 0x7f1c00103e30 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.405+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec017650 con 0x7f1c00103e30 2024-04-26T18:28:25.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.405+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a87d0 con 0x7f1c00103e30 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.406+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0006a110 con 0x7f1c00103e30 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.407+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1bec029070 con 0x7f1c00103e30 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.407+0000 7f1bfdffb700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 0x7f1bf00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.408+0000 7f1c04ae0700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 0x7f1bf00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.408+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1bec0d1760 con 0x7f1c00103e30 2024-04-26T18:28:25.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.408+0000 7f1c04ae0700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 0x7f1bf00a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c0019cc00 tx=0x7f1bf4009260 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.411+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bec013070 con 0x7f1c00103e30 2024-04-26T18:28:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.697+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0002e490 con 0x7f1c00103e30 2024-04-26T18:28:25.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.698+0000 7f1bfdffb700 1 -- 172.21.15.37:0/1277272979 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11886 (secure 0 0 0) 0x7f1bec09de20 con 0x7f1c00103e30 2024-04-26T18:28:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:28:25.699 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:26:35.544664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.700+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 msgr2=0x7f1bf00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 0x7f1bf00a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1c0019cc00 tx=0x7f1bf4009260 comp rx=0 tx=0).stop 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 msgr2=0x7f1c001a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0057c0 tx=0x7f1bec009580 comp rx=0 tx=0).stop 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 shutdown_connections 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c001094c0 0x7f1c0019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c00103470 0x7f1c001a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1bf00a3db0 0x7f1bf00a6270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 --2- 172.21.15.37:0/1277272979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1c00103e30 0x7f1c001a1e40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 >> 172.21.15.37:0/1277272979 conn(0x7f1c0007a580 msgr2=0x7f1c0010e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 shutdown_connections 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:25.701+0000 7f1c06d44700 1 -- 172.21.15.37:0/1277272979 wait complete. 2024-04-26T18:28:25.702 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-26T18:28:26.467 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:28:26.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:28:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:26 smithi037 ceph-mon[29171]: pgmap v126: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 829 op/s 2024-04-26T18:28:26.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:26 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1277272979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:27.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:26 smithi120 ceph-mon[32244]: pgmap v126: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 829 op/s 2024-04-26T18:28:27.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:26 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1277272979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:26 smithi170 ceph-mon[32697]: pgmap v126: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 829 op/s 2024-04-26T18:28:27.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:26 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1277272979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2158632697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae01023d0 msgr2=0x7f0ae0110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2158632697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae01023d0 0x7f0ae0110360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800b0a0 tx=0x7f0ad800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2158632697 shutdown_connections 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2158632697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae01023d0 0x7f0ae0110360 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2158632697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0109520 0x7f0ae0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2158632697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae0101a10 0x7f0ae0101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2158632697 >> 172.21.15.37:0/2158632697 conn(0x7f0ae00789c0 msgr2=0x7f0ae00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2158632697 shutdown_connections 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.650+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2158632697 wait complete. 2024-04-26T18:28:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 Processor -- start 2024-04-26T18:28:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 -- start start 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 0x7f0ae019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 0x7f0ae01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113660 con 0x7f0ae0109520 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01134e0 con 0x7f0ae0101a10 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ae57ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01137e0 con 0x7f0ae01023d0 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 0x7f0ae01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 0x7f0ae01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44314/0 (socket says 172.21.15.37:44314) 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0adf7fe700 1 -- 172.21.15.37:0/2589349870 learned_addr learned my addr 172.21.15.37:0/2589349870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 0x7f0ae019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.651+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:43028/0 (socket says 172.21.15.37:43028) 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 -- 172.21.15.37:0/2589349870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 msgr2=0x7f0ae019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 0x7f0ae019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 -- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 msgr2=0x7f0ae01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 0x7f0ae01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 -- 172.21.15.37:0/2589349870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae019a040 con 0x7f0ae0101a10 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 0x7f0ae019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000e9f0 tx=0x7f0ad000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ac7fff700 1 -- 172.21.15.37:0/2589349870 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000c9c0 con 0x7f0ae0101a10 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae019a330 con 0x7f0ae0101a10 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ac7fff700 1 -- 172.21.15.37:0/2589349870 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad000cb60 con 0x7f0ae0101a10 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ac7fff700 1 -- 172.21.15.37:0/2589349870 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0018680 con 0x7f0ae0101a10 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.652+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a65f0 con 0x7f0ae0101a10 2024-04-26T18:28:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.654+0000 7f0ac7fff700 1 -- 172.21.15.37:0/2589349870 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ad0010430 con 0x7f0ae0101a10 2024-04-26T18:28:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.655+0000 7f0ac7fff700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 0x7f0ac80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.655+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 0x7f0ac80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.655+0000 7f0ac7fff700 1 -- 172.21.15.37:0/2589349870 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0ad00d08b0 con 0x7f0ae0101a10 2024-04-26T18:28:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 0x7f0ac80a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0107b40 tx=0x7f0ad4009260 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 Processor -- start 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 -- start start 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c20610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa16b738d0 con 0x55aa16c2ba00 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa16b12770 con 0x55aa16c20230 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.656+0000 7f0af0040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aa16b6f890 con 0x55aa16c20b50 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c20610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44318/0 (socket says 172.21.15.37:44318) 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adf7fe700 1 -- 172.21.15.37:0/4148004539 learned_addr learned my addr 172.21.15.37:0/4148004539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:28.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:35034/0 (socket says 172.21.15.37:35034) 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4148004539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 msgr2=0x55aa16c20610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c20610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4148004539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 msgr2=0x55aa16c2ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4148004539 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa16ac0a60 con 0x55aa16c20b50 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adf7fe700 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0adeffd700 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c20610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40042f0 tx=0x7f0ad4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a7e17234e6848e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ac57fa700 1 -- 172.21.15.37:0/4148004539 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4003d10 con 0x55aa16c20b50 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ac57fa700 1 -- 172.21.15.37:0/4148004539 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40268b0 con 0x55aa16c20b50 2024-04-26T18:28:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.657+0000 7f0ac57fa700 1 -- 172.21.15.37:0/4148004539 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad401e6e0 con 0x55aa16c20b50 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 -- 172.21.15.37:0/4148004539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 msgr2=0x55aa16c2b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40042f0 tx=0x7f0ad4014000 comp rx=0 tx=0).stop 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 -- 172.21.15.37:0/4148004539 shutdown_connections 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c20610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4148004539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20b50 0x55aa16c2b4c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 -- 172.21.15.37:0/4148004539 >> 172.21.15.37:0/4148004539 conn(0x55aa16b4ded0 msgr2=0x55aa16c1c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 -- 172.21.15.37:0/4148004539 shutdown_connections 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.658+0000 7f0af0040b80 1 -- 172.21.15.37:0/4148004539 wait complete. 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 -- start start 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 Processor -- start 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20230 0x55aa16c2b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c28a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa16c2e5d0 con 0x55aa16c2ba00 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55aa16ad9ae0 con 0x55aa16c20b50 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aa16c2e750 con 0x55aa16c20230 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0af0040b80 10 client.0 _ll_register_callbacks cb 0x55aa16b07710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c28a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:43052/0 (socket says 172.21.15.37:43052) 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c28a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44330/0 (socket says 172.21.15.37:44330) 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4012182386 learned_addr learned my addr 172.21.15.37:0/4012182386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.659+0000 7f0adeffd700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20230 0x55aa16c2b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4012182386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20230 msgr2=0x55aa16c2b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20230 0x55aa16c2b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4012182386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 msgr2=0x55aa16c28a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c28a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 -- 172.21.15.37:0/4012182386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa16c29110 con 0x55aa16c20b50 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4005920 tx=0x7f0ad4003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb4ec264b27bc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ac2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4026c70 con 0x55aa16c20b50 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0ad4005050 con 0x55aa16c20b50 2024-04-26T18:28:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad40268b0 con 0x55aa16c20b50 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ad4005250 con 0x55aa16c20b50 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0af0040b80 10 client.25394 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa16c390d0 con 0x55aa16c20b50 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.660+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad40268b0 con 0x55aa16c20b50 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ad4030420 con 0x55aa16c20b50 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 10 client.25394 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 20 client.25394 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16c3b650 0x55aa16c3da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 4 client.25394 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa16b812b0 con 0x55aa16c3b650 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0adeffd700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16c3b650 0x55aa16c3da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0ac2ff5700 10 client.25394 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.661+0000 7f0adeffd700 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16c3b650 0x55aa16c3da40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a121c4dbdb81ec6b server_cookie=5c73619244499147 in_seq=0 out_seq=0 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/4012182386 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa16b812b0 con 0x55aa16c3b650 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0ac2ff5700 10 client.25394 handle_command_reply: tid=0 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 client.25394 shutdown 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 msgr2=0x55aa16c284d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4005920 tx=0x7f0ad4003d40 comp rx=0 tx=0).stop 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 shutdown_connections 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16c3b650 0x55aa16c3da40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c2ba00 0x55aa16c28a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0ac2ff5700 0 client.25394 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c20230 0x55aa16c2b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 --2- 172.21.15.37:0/4012182386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20b50 0x55aa16c284d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 >> 172.21.15.37:0/4012182386 conn(0x55aa16b4ded0 msgr2=0x55aa16c1c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 shutdown_connections 2024-04-26T18:28:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.662+0000 7f0af0040b80 1 -- 172.21.15.37:0/4012182386 wait complete. 2024-04-26T18:28:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.663+0000 7f0af0040b80 20 client.25394 trim_cache size 0 max 16384 2024-04-26T18:28:28.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.686+0000 7f0af0040b80 1 Processor -- start 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.686+0000 7f0af0040b80 1 -- start start 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.686+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.686+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.686+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0af0040b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ae3c0 con 0x55aa16c20b50 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0af0040b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01abc50 con 0x55aa16c20230 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0af0040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01a9ac0 con 0x55aa16c2ba00 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44332/0 (socket says 172.21.15.37:44332) 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 -- 172.21.15.37:0/455161759 learned_addr learned my addr 172.21.15.37:0/455161759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0adf7fe700 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 -- 172.21.15.37:0/455161759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 msgr2=0x55aa16c37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 -- 172.21.15.37:0/455161759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 msgr2=0x55aa16c43090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c43090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 -- 172.21.15.37:0/455161759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa16c13fd0 con 0x55aa16c20b50 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40309e0 tx=0x7f0ad4026990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38a48c9e93bd1de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.687+0000 7f0ac4ff9700 1 -- 172.21.15.37:0/455161759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad403c030 con 0x55aa16c20b50 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0ac4ff9700 1 -- 172.21.15.37:0/455161759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad4030420 con 0x55aa16c20b50 2024-04-26T18:28:28.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0ac4ff9700 1 -- 172.21.15.37:0/455161759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40053f0 con 0x55aa16c20b50 2024-04-26T18:28:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 -- 172.21.15.37:0/455161759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 msgr2=0x55aa16c435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40309e0 tx=0x7f0ad4026990 comp rx=0 tx=0).stop 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 -- 172.21.15.37:0/455161759 shutdown_connections 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c43090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 --2- 172.21.15.37:0/455161759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c20b50 0x55aa16c435d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 -- 172.21.15.37:0/455161759 >> 172.21.15.37:0/455161759 conn(0x55aa16b4ded0 msgr2=0x55aa16c26de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 -- 172.21.15.37:0/455161759 shutdown_connections 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.688+0000 7f0af0040b80 1 -- 172.21.15.37:0/455161759 wait complete. 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 -- start start 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 Processor -- start 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01aa110 con 0x55aa16c9e8f0 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ab580 con 0x55aa16c20230 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01aa7e0 con 0x55aa16c2ba00 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0af0040b80 10 client.0 _ll_register_callbacks cb 0x55aa16b07710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0adf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44342/0 (socket says 172.21.15.37:44342) 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0adeffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0adf7fe700 1 -- 172.21.15.37:0/2107923895 learned_addr learned my addr 172.21.15.37:0/2107923895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.689+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 -- 172.21.15.37:0/2107923895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 msgr2=0x55aa16c2ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 -- 172.21.15.37:0/2107923895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 msgr2=0x55aa16ca0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 -- 172.21.15.37:0/2107923895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa16ca1220 con 0x55aa16c20230 2024-04-26T18:28:28.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adf7fe700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:28.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0adeffd700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c2fa60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00cdf90 tx=0x7f0ad001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6971f7f44cd53cef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0ac2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00a5450 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0ad00870b0 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad4030420 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ad00ce210 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0af0040b80 10 client.25400 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.690+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa16ca2530 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad4030420 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f0ad00b3070 con 0x55aa16c20230 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 10 client.25400 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 10 client.25400 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 20 client.25400 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16ca2cb0 0x55aa16ca50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 4 client.25400 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:28:28.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x55aa16b7b350 con 0x55aa16ca2cb0 2024-04-26T18:28:28.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.691+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16ca2cb0 0x55aa16ca50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.692+0000 7f0ade7fc700 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16ca2cb0 0x55aa16ca50d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d34bbae4d4ae854 server_cookie=5a602fdad84c499b in_seq=0 out_seq=0 2024-04-26T18:28:28.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:28.692+0000 7f0ac2ff5700 10 client.25400 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:28.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:28 smithi037 ceph-mon[29171]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:29.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:28 smithi120 ceph-mon[32244]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:29.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:28 smithi170 ceph-mon[32697]: pgmap v127: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:29.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:29 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:30.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:29 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:30.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:29 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:30 smithi037 ceph-mon[29171]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:30 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:30 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:30.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:30 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:28:31.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:30 smithi120 ceph-mon[32244]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:31.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:30 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:31.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:30 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:31.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:30 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:28:31.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:30 smithi170 ceph-mon[32697]: pgmap v128: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2024-04-26T18:28:31.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:30 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:31.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:30 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:31.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:30 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.828+0000 7f0ac2ff5700 1 -- 172.21.15.37:0/2107923895 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55aa16b7b350 con 0x55aa16ca2cb0 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.828+0000 7f0ac2ff5700 10 client.25400 handle_command_reply: tid=0 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 client.25400 shutdown 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 msgr2=0x55aa16c2fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c2fa60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00cdf90 tx=0x7f0ad001a040 comp rx=0 tx=0).stop 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 shutdown_connections 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55aa16ca2cb0 0x55aa16ca50d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa16c9e8f0 0x55aa16ca0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55aa16c20230 0x55aa16c2fa60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0ac2ff5700 0 client.25400 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 --2- 172.21.15.37:0/2107923895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aa16c2ba00 0x55aa16c2ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 >> 172.21.15.37:0/2107923895 conn(0x55aa16b4ded0 msgr2=0x55aa16c29d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 shutdown_connections 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 1 -- 172.21.15.37:0/2107923895 wait complete. 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.829+0000 7f0af0040b80 20 client.25400 trim_cache size 0 max 16384 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:28:31.831 INFO:teuthology.orchestra.run.smithi037.stdout: "4c5b6836": { 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 79.5, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:28:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 msgr2=0x7f0ac80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 0x7f0ac80a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0107b40 tx=0x7f0ad4009260 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 msgr2=0x7f0ae019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000e9f0 tx=0x7f0ad000ed00 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 shutdown_connections 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ae0109520 0x7f0ae01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ac80a3ed0 0x7f0ac80a6390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae0101a10 0x7f0ae019f720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 --2- 172.21.15.37:0/2589349870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ae01023d0 0x7f0ae019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 >> 172.21.15.37:0/2589349870 conn(0x7f0ae00789c0 msgr2=0x7f0ae01061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.830+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 shutdown_connections 2024-04-26T18:28:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:31.831+0000 7f0ae57ae700 1 -- 172.21.15.37:0/2589349870 wait complete. 2024-04-26T18:28:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- 172.21.15.37:0/466750763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 msgr2=0x7f40b410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 --2- 172.21.15.37:0/466750763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b410c9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f409c009900 tx=0x7f409c02a510 comp rx=0 tx=0).stop 2024-04-26T18:28:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- 172.21.15.37:0/466750763 shutdown_connections 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 --2- 172.21.15.37:0/466750763 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 0x7f40b410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 --2- 172.21.15.37:0/466750763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b410c9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 --2- 172.21.15.37:0/466750763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- 172.21.15.37:0/466750763 >> 172.21.15.37:0/466750763 conn(0x7f40b40789b0 msgr2=0x7f40b4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- 172.21.15.37:0/466750763 shutdown_connections 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- 172.21.15.37:0/466750763 wait complete. 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 Processor -- start 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.433+0000 7f40bad71640 1 -- start start 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 0x7f40b419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112c00 con 0x7f40b41044e0 2024-04-26T18:28:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112a80 con 0x7f40b410cf00 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40bad71640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112d80 con 0x7f40b4103bc0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44356/0 (socket says 172.21.15.37:44356) 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 -- 172.21.15.37:0/2372520203 learned_addr learned my addr 172.21.15.37:0/2372520203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b930e640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 0x7f40b419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 -- 172.21.15.37:0/2372520203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 msgr2=0x7f40b4199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 -- 172.21.15.37:0/2372520203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 msgr2=0x7f40b419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 0x7f40b419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 -- 172.21.15.37:0/2372520203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b419e050 con 0x7f40b41044e0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.434+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f409c005ab0 tx=0x7f409c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.435+0000 7f40b1ffb640 1 -- 172.21.15.37:0/2372520203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c03f070 con 0x7f40b41044e0 2024-04-26T18:28:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.435+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b419e2e0 con 0x7f40b41044e0 2024-04-26T18:28:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.435+0000 7f40b1ffb640 1 -- 172.21.15.37:0/2372520203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c031d60 con 0x7f40b41044e0 2024-04-26T18:28:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.435+0000 7f40b1ffb640 1 -- 172.21.15.37:0/2372520203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c035030 con 0x7f40b41044e0 2024-04-26T18:28:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.435+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a5bd0 con 0x7f40b41044e0 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.437+0000 7f40b1ffb640 1 -- 172.21.15.37:0/2372520203 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f409c042070 con 0x7f40b41044e0 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 Processor -- start 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 -- start start 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b1ffb640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 0x7f40a00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b42b9d460 con 0x564b42c37820 2024-04-26T18:28:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564b42b57590 con 0x564b42c420e0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40bdb67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564b42b99420 con 0x564b42c36f00 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 0x7f40a00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44362/0 (socket says 172.21.15.37:44362) 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 -- 172.21.15.37:0/2768065317 learned_addr learned my addr 172.21.15.37:0/2768065317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b930e640 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b1ffb640 1 -- 172.21.15.37:0/2372520203 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f409c0f10d0 con 0x7f40b41044e0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b930e640 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 -- 172.21.15.37:0/2768065317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 msgr2=0x564b42c372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 -- 172.21.15.37:0/2768065317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 msgr2=0x564b42c444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.438+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40b3fff640 1 -- 172.21.15.37:0/2768065317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b42b640d0 con 0x564b42c37820 2024-04-26T18:28:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 0x7f40a00a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40a40058c0 tx=0x7f40a4005850 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f409c0ee870 tx=0x7f409c03b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279fa91b27b05ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40aaffd640 1 -- 172.21.15.37:0/2768065317 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c049070 con 0x564b42c37820 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40aaffd640 1 -- 172.21.15.37:0/2768065317 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c03be50 con 0x564b42c37820 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40aaffd640 1 -- 172.21.15.37:0/2768065317 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0c5cd0 con 0x564b42c37820 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40bdb67740 1 -- 172.21.15.37:0/2768065317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 msgr2=0x564b42c41ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.439+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f409c0ee870 tx=0x7f409c03b920 comp rx=0 tx=0).stop 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 -- 172.21.15.37:0/2768065317 shutdown_connections 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c41ba0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2768065317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 -- 172.21.15.37:0/2768065317 >> 172.21.15.37:0/2768065317 conn(0x564b42c33410 msgr2=0x564b42c337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 -- 172.21.15.37:0/2768065317 shutdown_connections 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 -- 172.21.15.37:0/2768065317 wait complete. 2024-04-26T18:28:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 -- start start 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 Processor -- start 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.440+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 0x564b42c3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c37820 0x564b42c3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b42c44ad0 con 0x564b42c420e0 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564b42b24b60 con 0x564b42c36f00 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564b42c44c50 con 0x564b42c37820 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 10 client.0 _ll_register_callbacks cb 0x564b42b492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44376/0 (socket says 172.21.15.37:44376) 2024-04-26T18:28:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 -- 172.21.15.37:0/1473733201 learned_addr learned my addr 172.21.15.37:0/1473733201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 0x564b42c3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b3fff640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c37820 0x564b42c3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 -- 172.21.15.37:0/1473733201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c37820 msgr2=0x564b42c3f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c37820 0x564b42c3f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 -- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 msgr2=0x564b42c3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 0x564b42c3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 -- 172.21.15.37:0/1473733201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b428af210 con 0x564b42c420e0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 0x564b42c3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40b930e640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40ac012ea0 tx=0x7f40ac012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd16ce15f423314 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f4093fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac003b70 con 0x564b42c420e0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f40ac003d10 con 0x564b42c420e0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.441+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409c03be50 con 0x564b42c420e0 2024-04-26T18:28:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40ac024050 con 0x564b42c420e0 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 10 client.15786 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b4297b620 con 0x564b42c420e0 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c03be50 con 0x564b42c420e0 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f40ac00f840 con 0x564b42c420e0 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 10 client.15786 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 20 client.15786 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42c52070 0x564b42c54460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 4 client.15786 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b42bab070 con 0x564b42c52070 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.442+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42c52070 0x564b42c54460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.443+0000 7f4093fff640 10 client.15786 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.443+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42c52070 0x564b42c54460 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bae69dd80be6c0 server_cookie=f4827d03f3b4fc19 in_seq=0 out_seq=0 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f4093fff640 1 -- 172.21.15.37:0/1473733201 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b42bab070 con 0x564b42c52070 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f4093fff640 10 client.15786 handle_command_reply: tid=0 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 client.15786 shutdown 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 msgr2=0x564b42c3fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f40ac012ea0 tx=0x7f40ac012f90 comp rx=0 tx=0).stop 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 shutdown_connections 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42c52070 0x564b42c54460 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3fb30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c37820 0x564b42c3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f4093fff640 0 client.15786 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1473733201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c36f00 0x564b42c3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 >> 172.21.15.37:0/1473733201 conn(0x564b42c33410 msgr2=0x564b42c39190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 shutdown_connections 2024-04-26T18:28:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 1 -- 172.21.15.37:0/1473733201 wait complete. 2024-04-26T18:28:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.444+0000 7f40bdb67740 20 client.15786 trim_cache size 0 max 16384 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 Processor -- start 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 -- start start 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41b0dc0 con 0x564b42c37820 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ae690 con 0x564b42c420e0 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40bdb67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ac4e0 con 0x564b42c36f00 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44380/0 (socket says 172.21.15.37:44380) 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.459+0000 7f40b3fff640 1 -- 172.21.15.37:0/1369000926 learned_addr learned my addr 172.21.15.37:0/1369000926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:32.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b930e640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 -- 172.21.15.37:0/1369000926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 msgr2=0x564b42c4d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c4d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 -- 172.21.15.37:0/1369000926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 msgr2=0x564b42c59d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c59d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 -- 172.21.15.37:0/1369000926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b42c54ae0 con 0x564b42c37820 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b930e640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40b3fff640 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f409c0bcd60 tx=0x7f409c0bcca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba13020e838ecfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40aa7fc640 1 -- 172.21.15.37:0/1369000926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c049070 con 0x564b42c37820 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40aa7fc640 1 -- 172.21.15.37:0/1369000926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c03be50 con 0x564b42c37820 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.460+0000 7f40aa7fc640 1 -- 172.21.15.37:0/1369000926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0c11f0 con 0x564b42c37820 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- 172.21.15.37:0/1369000926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 msgr2=0x564b42c398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f409c0bcd60 tx=0x7f409c0bcca0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- 172.21.15.37:0/1369000926 shutdown_connections 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c420e0 0x564b42c59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c37820 0x564b42c398c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 --2- 172.21.15.37:0/1369000926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c4d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- 172.21.15.37:0/1369000926 >> 172.21.15.37:0/1369000926 conn(0x564b42c33410 msgr2=0x564b42c325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- 172.21.15.37:0/1369000926 shutdown_connections 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- 172.21.15.37:0/1369000926 wait complete. 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 -- start start 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.461+0000 7f40bdb67740 1 Processor -- start 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c3beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c37820 0x564b42c3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41acb50 con 0x564b42c420e0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41adfc0 con 0x564b42c37820 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41ad220 con 0x564b42c36f00 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 10 client.0 _ll_register_callbacks cb 0x7f40ac012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44384/0 (socket says 172.21.15.37:44384) 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 -- 172.21.15.37:0/2666953311 learned_addr learned my addr 172.21.15.37:0/2666953311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b3fff640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c37820 0x564b42c3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c3beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 -- 172.21.15.37:0/2666953311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 msgr2=0x564b42c3beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c3beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 -- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c37820 msgr2=0x564b42c3c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c37820 0x564b42c3c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 -- 172.21.15.37:0/2666953311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b42cbbaf0 con 0x564b42c420e0 2024-04-26T18:28:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40b930e640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00fc60 tx=0x7f40ac007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7db9a6fe5a8949 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f4093fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac02c020 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f40ac00f630 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.462+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409c03be50 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40ac037050 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 10 client.15792 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b42cbcde0 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c03be50 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f40ac0054f0 con 0x564b42c420e0 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 20 client.15792 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42cbd5e0 0x564b42cbfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 4 client.15792 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b42ba4ee0 con 0x564b42cbd5e0 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.463+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42cbd5e0 0x564b42cbfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.464+0000 7f4093fff640 10 client.15792 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.464+0000 7f40b8b0d640 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42cbd5e0 0x564b42cbfa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994dec963099cfcd server_cookie=652500d3ac2027a1 in_seq=0 out_seq=0 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f4093fff640 1 -- 172.21.15.37:0/2666953311 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b42ba4ee0 con 0x564b42cbd5e0 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f4093fff640 10 client.15792 handle_command_reply: tid=0 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 client.15792 shutdown 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 msgr2=0x564b42c3c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00fc60 tx=0x7f40ac007f10 comp rx=0 tx=0).stop 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 shutdown_connections 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564b42cbd5e0 0x564b42cbfa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b42c37820 0x564b42c3c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b42c420e0 0x564b42c3c930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f4093fff640 0 client.15792 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 --2- 172.21.15.37:0/2666953311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b42c36f00 0x564b42c3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 >> 172.21.15.37:0/2666953311 conn(0x564b42c33410 msgr2=0x564b42c3f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 shutdown_connections 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 1 -- 172.21.15.37:0/2666953311 wait complete. 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.482+0000 7f40bdb67740 20 client.15792 trim_cache size 0 max 16384 2024-04-26T18:28:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:28:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:28:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:28:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 msgr2=0x7f40a00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 0x7f40a00a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40a40058c0 tx=0x7f40a4005850 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 msgr2=0x7f40b41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f409c005ab0 tx=0x7f409c003cd0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 shutdown_connections 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f40a00a3dc0 0x7f40a00a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f40b410cf00 0x7f40b419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40b41044e0 0x7f40b41995a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 --2- 172.21.15.37:0/2372520203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40b4103bc0 0x7f40b4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 >> 172.21.15.37:0/2372520203 conn(0x7f40b40789b0 msgr2=0x7f40b40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 shutdown_connections 2024-04-26T18:28:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:32.483+0000 7f40bad71640 1 -- 172.21.15.37:0/2372520203 wait complete. 2024-04-26T18:28:32.629 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4c5b6836', quiesce took 9.4 seconds. Will release after: 19.5 2024-04-26T18:28:32.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:32 smithi037 ceph-mon[29171]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 713 op/s 2024-04-26T18:28:33.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:32 smithi120 ceph-mon[32244]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 713 op/s 2024-04-26T18:28:33.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:32 smithi170 ceph-mon[32697]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 713 op/s 2024-04-26T18:28:34.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:34 smithi037 ceph-mon[29171]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-04-26T18:28:35.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:34 smithi120 ceph-mon[32244]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-04-26T18:28:35.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:34 smithi170 ceph-mon[32697]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-04-26T18:28:36.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:36 smithi037 ceph-mon[29171]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 502 op/s 2024-04-26T18:28:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:36 smithi120 ceph-mon[32244]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 502 op/s 2024-04-26T18:28:37.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:36 smithi170 ceph-mon[32697]: pgmap v131: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 502 op/s 2024-04-26T18:28:38.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:38 smithi037 ceph-mon[29171]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:39.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:38 smithi120 ceph-mon[32244]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:39.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:38 smithi170 ceph-mon[32697]: pgmap v132: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:40.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:40 smithi037 ceph-mon[29171]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:41.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:40 smithi120 ceph-mon[32244]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:41.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:40 smithi170 ceph-mon[32697]: pgmap v133: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 240 op/s 2024-04-26T18:28:42.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:42 smithi037 ceph-mon[29171]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-04-26T18:28:43.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:42 smithi120 ceph-mon[32244]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-04-26T18:28:43.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:42 smithi170 ceph-mon[32697]: pgmap v134: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-04-26T18:28:44.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:44 smithi037 ceph-mon[29171]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 256 KiB/s wr, 71 op/s 2024-04-26T18:28:45.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:44 smithi120 ceph-mon[32244]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 256 KiB/s wr, 71 op/s 2024-04-26T18:28:45.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:44 smithi170 ceph-mon[32697]: pgmap v135: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 256 KiB/s wr, 71 op/s 2024-04-26T18:28:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 -- 172.21.15.37:0/1701612945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18102210 msgr2=0x7f1d1810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1701612945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18102210 0x7f1d1810e720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009cd0 tx=0x7f1d1002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 -- 172.21.15.37:0/1701612945 shutdown_connections 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1701612945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18101850 0x7f1d18101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1701612945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18102210 0x7f1d1810e720 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1701612945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18107860 0x7f1d18107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 -- 172.21.15.37:0/1701612945 >> 172.21.15.37:0/1701612945 conn(0x7f1d18077bb0 msgr2=0x7f1d18077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 -- 172.21.15.37:0/1701612945 shutdown_connections 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.585+0000 7f1d1da47640 1 -- 172.21.15.37:0/1701612945 wait complete. 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 Processor -- start 2024-04-26T18:28:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 -- start start 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 0x7f1d1819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18107860 0x7f1d181a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18111b20 con 0x7f1d18101850 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181119a0 con 0x7f1d18107860 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d1da47640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1d18111ca0 con 0x7f1d18102210 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.586+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48814/0 (socket says 172.21.15.37:48814) 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 -- 172.21.15.37:0/1626754201 learned_addr learned my addr 172.21.15.37:0/1626754201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d177fe640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18107860 0x7f1d181a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d167fc640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 0x7f1d1819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 -- 172.21.15.37:0/1626754201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 msgr2=0x7f1d1819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 0x7f1d1819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 -- 172.21.15.37:0/1626754201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18107860 msgr2=0x7f1d181a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18107860 0x7f1d181a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 -- 172.21.15.37:0/1626754201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d181a5420 con 0x7f1d18101850 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d167fc640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 0x7f1d1819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d1ca45640 1 -- 172.21.15.37:0/1626754201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800cd80 con 0x7f1d18101850 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d1ca45640 1 -- 172.21.15.37:0/1626754201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d080054d0 con 0x7f1d18101850 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d181a55f0 con 0x7f1d18101850 2024-04-26T18:28:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d1ca45640 1 -- 172.21.15.37:0/1626754201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08010690 con 0x7f1d18101850 2024-04-26T18:28:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.587+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a5b10 con 0x7f1d18101850 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d1ca45640 1 -- 172.21.15.37:0/1626754201 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d08004e90 con 0x7f1d18101850 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 Processor -- start 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 -- start start 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32896ef00 0x55e32896f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32897a0e0 0x55e32897c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e3288d5460 con 0x55e32896f820 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e32888f590 con 0x55e32897a0e0 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.590+0000 7f1d203fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e3288d1420 con 0x55e32896ef00 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48816/0 (socket says 172.21.15.37:48816) 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 -- 172.21.15.37:0/775301519 learned_addr learned my addr 172.21.15.37:0/775301519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d1ca45640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 0x7f1d000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d177fe640 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32896ef00 0x55e32896f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d167fc640 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32897a0e0 0x55e32897c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d177fe640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 0x7f1d000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d1ca45640 1 -- 172.21.15.37:0/1626754201 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1d080d0c40 con 0x7f1d18101850 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 -- 172.21.15.37:0/775301519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32896ef00 msgr2=0x55e32896f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32896ef00 0x55e32896f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 -- 172.21.15.37:0/775301519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32897a0e0 msgr2=0x55e32897c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32897a0e0 0x55e32897c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 -- 172.21.15.37:0/775301519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32889c0d0 con 0x55e32896f820 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d08099540 tx=0x7f1d08004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1eb5c1a3e5ee3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1d177fe640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 0x7f1d000a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d10004af0 tx=0x7f1d10005da0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.591+0000 7f1cfdffb640 1 -- 172.21.15.37:0/775301519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0809c6b0 con 0x55e32896f820 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1cfdffb640 1 -- 172.21.15.37:0/775301519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0809d270 con 0x55e32896f820 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1cfdffb640 1 -- 172.21.15.37:0/775301519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08005180 con 0x55e32896f820 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 -- 172.21.15.37:0/775301519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 msgr2=0x55e328979ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d08099540 tx=0x7f1d08004a90 comp rx=0 tx=0).stop 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 -- 172.21.15.37:0/775301519 shutdown_connections 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32897a0e0 0x55e32897c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328979ba0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 --2- 172.21.15.37:0/775301519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32896ef00 0x55e32896f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 -- 172.21.15.37:0/775301519 >> 172.21.15.37:0/775301519 conn(0x55e32896b410 msgr2=0x55e32896b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 -- 172.21.15.37:0/775301519 shutdown_connections 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.592+0000 7f1d203fd740 1 -- 172.21.15.37:0/775301519 wait complete. 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 -- start start 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 Processor -- start 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e328977110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328977b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e32897cc50 con 0x55e32896f820 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e32885cb60 con 0x55e32896ef00 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e32897cdd0 con 0x55e32897a0e0 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d203fd740 10 client.0 _ll_register_callbacks cb 0x55e3288812b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48820/0 (socket says 172.21.15.37:48820) 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.593+0000 7f1d167fc640 1 -- 172.21.15.37:0/3186477406 learned_addr learned my addr 172.21.15.37:0/3186477406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d177fe640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328977b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e328977110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 -- 172.21.15.37:0/3186477406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 msgr2=0x55e328977b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328977b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 -- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 msgr2=0x55e328977110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e328977110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 -- 172.21.15.37:0/3186477406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3285e7210 con 0x55e32896f820 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d177fe640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328977b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00bfb0 tx=0x7f1d0c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3b03dac6089e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0035f0 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1d0c003790 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0809d270 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c003990 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d203fd740 10 client.15804 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.594+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3286b3620 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0809d270 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d0c019070 con 0x55e32896f820 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 20 client.15804 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e32898a310 0x55e32898c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 4 client.15804 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3288e3070 con 0x55e32898a310 2024-04-26T18:28:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.595+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e32898a310 0x55e32898c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1cf6ffd640 10 client.15804 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e32898a310 0x55e32898c790 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b196bed17a0889 server_cookie=473c5847789f6ad6 in_seq=0 out_seq=0 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3186477406 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3288e3070 con 0x55e32898a310 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1cf6ffd640 10 client.15804 handle_command_reply: tid=0 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1d203fd740 1 client.15804 shutdown 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 msgr2=0x55e328977650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.596+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00bfb0 tx=0x7f1d0c0128e0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 shutdown_connections 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e32898a310 0x55e32898c790 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328977b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e328977650 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1cf6ffd640 0 client.15804 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3186477406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e328977110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 >> 172.21.15.37:0/3186477406 conn(0x55e32896b410 msgr2=0x55e328971140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 shutdown_connections 2024-04-26T18:28:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 1 -- 172.21.15.37:0/3186477406 wait complete. 2024-04-26T18:28:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.597+0000 7f1d203fd740 20 client.15804 trim_cache size 0 max 16384 2024-04-26T18:28:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 Processor -- start 2024-04-26T18:28:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 -- start start 2024-04-26T18:28:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32896ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328979050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181b0dd0 con 0x55e32896f820 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ae6a0 con 0x55e32896ef00 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d203fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ac4f0 con 0x55e32897a0e0 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48828/0 (socket says 172.21.15.37:48828) 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.612+0000 7f1d167fc640 1 -- 172.21.15.37:0/3637287269 learned_addr learned my addr 172.21.15.37:0/3637287269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d177fe640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328979050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32896ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 -- 172.21.15.37:0/3637287269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 msgr2=0x55e328979050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328979050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 -- 172.21.15.37:0/3637287269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 msgr2=0x55e32896ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32896ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 -- 172.21.15.37:0/3637287269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3289701a0 con 0x55e32896f820 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d177fe640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328979050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32896ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a704365d6f16cba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1cfd7fa640 1 -- 172.21.15.37:0/3637287269 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030070 con 0x55e32896f820 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1cfd7fa640 1 -- 172.21.15.37:0/3637287269 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0c0035f0 con 0x55e32896f820 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.613+0000 7f1cfd7fa640 1 -- 172.21.15.37:0/3637287269 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0054e0 con 0x55e32896f820 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- 172.21.15.37:0/3637287269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 msgr2=0x55e3289718c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c005990 comp rx=0 tx=0).stop 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- 172.21.15.37:0/3637287269 shutdown_connections 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e328979050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e3289718c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3637287269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32896ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- 172.21.15.37:0/3637287269 >> 172.21.15.37:0/3637287269 conn(0x55e32896b410 msgr2=0x55e328978940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- 172.21.15.37:0/3637287269 shutdown_connections 2024-04-26T18:28:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- 172.21.15.37:0/3637287269 wait complete. 2024-04-26T18:28:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.614+0000 7f1d203fd740 1 -- start start 2024-04-26T18:28:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 Processor -- start 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32897daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e32897e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181acb60 con 0x55e32896f820 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181adfd0 con 0x55e32896ef00 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1d181ad230 con 0x55e32897a0e0 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d203fd740 10 client.0 _ll_register_callbacks cb 0x7f1d0c00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48834/0 (socket says 172.21.15.37:48834) 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 -- 172.21.15.37:0/3835870091 learned_addr learned my addr 172.21.15.37:0/3835870091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32897daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 -- 172.21.15.37:0/3835870091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 msgr2=0x55e32897e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e32897e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 -- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 msgr2=0x55e32897daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32897daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 -- 172.21.15.37:0/3835870091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3289ef920 con 0x55e32896f820 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1d167fc640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c012920 tx=0x7f1d0c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66500e590a7d61eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.615+0000 7f1cf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c030040 con 0x55e32896f820 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1d0c0053e0 con 0x55e32896f820 2024-04-26T18:28:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0c0035f0 con 0x55e32896f820 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c0037a0 con 0x55e32896f820 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 10 client.15810 fetch_fsmap learned FSMap version 27 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e3289f0c30 con 0x55e32896f820 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0c0035f0 con 0x55e32896f820 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f1d0c033070 con 0x55e32896f820 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 10 client.15810 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 20 client.15810 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e3289f1310 0x55e3289f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 4 client.15810 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3288dcee0 con 0x55e3289f1310 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.616+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e3289f1310 0x55e3289f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.617+0000 7f1d16ffd640 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e3289f1310 0x55e3289f3730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd21277dc2ce05ad server_cookie=cfd632f6b002446e in_seq=0 out_seq=0 2024-04-26T18:28:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.617+0000 7f1cf6ffd640 10 client.15810 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.632+0000 7f1cf6ffd640 1 -- 172.21.15.37:0/3835870091 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3288dcee0 con 0x55e3289f1310 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.632+0000 7f1cf6ffd640 10 client.15810 handle_command_reply: tid=0 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.632+0000 7f1d203fd740 1 client.15810 shutdown 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.632+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 msgr2=0x55e32897dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c012920 tx=0x7f1d0c004530 comp rx=0 tx=0).stop 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 shutdown_connections 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55e3289f1310 0x55e3289f3730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e32897a0e0 0x55e32897e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e32896f820 0x55e32897dfe0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 --2- 172.21.15.37:0/3835870091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e32896ef00 0x55e32897daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1cf6ffd640 0 client.15810 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 >> 172.21.15.37:0/3835870091 conn(0x55e32896b410 msgr2=0x55e3289791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 shutdown_connections 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 1 -- 172.21.15.37:0/3835870091 wait complete. 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d203fd740 20 client.15810 trim_cache size 0 max 16384 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:28:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 msgr2=0x7f1d000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 0x7f1d000a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d10004af0 tx=0x7f1d10005da0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 msgr2=0x7f1d18199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800e9f0 tx=0x7f1d0800edf0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 shutdown_connections 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f1d000a3e40 0x7f1d000a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d18107860 0x7f1d181a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d18102210 0x7f1d1819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 --2- 172.21.15.37:0/1626754201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d18101850 0x7f1d18199e60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.633+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 >> 172.21.15.37:0/1626754201 conn(0x7f1d18077bb0 msgr2=0x7f1d180ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.634+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 shutdown_connections 2024-04-26T18:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:45.634+0000 7f1d1da47640 1 -- 172.21.15.37:0/1626754201 wait complete. 2024-04-26T18:28:46.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:46 smithi037 ceph-mon[29171]: pgmap v136: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 258 KiB/s wr, 75 op/s 2024-04-26T18:28:47.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:46 smithi120 ceph-mon[32244]: pgmap v136: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 258 KiB/s wr, 75 op/s 2024-04-26T18:28:47.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:46 smithi170 ceph-mon[32697]: pgmap v136: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 258 KiB/s wr, 75 op/s 2024-04-26T18:28:48.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:48 smithi037 ceph-mon[29171]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:49.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:48 smithi120 ceph-mon[32244]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:49.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:48 smithi170 ceph-mon[32697]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:50.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:50 smithi037 ceph-mon[29171]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:51.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:50 smithi120 ceph-mon[32244]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:51.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:50 smithi170 ceph-mon[32697]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-26T18:28:51.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:51 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:28:52.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:51 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:28:52.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:51 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:28:52.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:52.837 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:52 smithi037 ceph-mon[29171]: pgmap v139: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-26T18:28:53.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:52 smithi120 ceph-mon[32244]: pgmap v139: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-26T18:28:53.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:52 smithi170 ceph-mon[32697]: pgmap v139: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-26T18:28:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.116+0000 7f3f09afe700 1 -- 172.21.15.37:0/3241303590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 msgr2=0x7f3f04103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.116+0000 7f3f09afe700 1 --2- 172.21.15.37:0/3241303590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f04103920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009990 tx=0x7f3ef8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 -- 172.21.15.37:0/3241303590 shutdown_connections 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 --2- 172.21.15.37:0/3241303590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 0x7f3f041102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 --2- 172.21.15.37:0/3241303590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 0x7f3f041098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 --2- 172.21.15.37:0/3241303590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f04103920 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 -- 172.21.15.37:0/3241303590 >> 172.21.15.37:0/3241303590 conn(0x7f3f0407a590 msgr2=0x7f3f0407a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 -- 172.21.15.37:0/3241303590 shutdown_connections 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 -- 172.21.15.37:0/3241303590 wait complete. 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.117+0000 7f3f09afe700 1 Processor -- start 2024-04-26T18:28:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 -- start start 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 0x7f3f0419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 0x7f3f04199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04113600 con 0x7f3f041034a0 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04113480 con 0x7f3f04103e60 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f09afe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04113780 con 0x7f3f041094e0 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48848/0 (socket says 172.21.15.37:48848) 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f037fe700 1 -- 172.21.15.37:0/353541587 learned_addr learned my addr 172.21.15.37:0/353541587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:54.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f03fff700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 0x7f3f04199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.118+0000 7f3f02ffd700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 0x7f3f0419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 -- 172.21.15.37:0/353541587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 msgr2=0x7f3f04199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 0x7f3f04199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 -- 172.21.15.37:0/353541587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 msgr2=0x7f3f0419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 0x7f3f0419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 -- 172.21.15.37:0/353541587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0419a080 con 0x7f3f041034a0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f037fe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400d980 tx=0x7f3ef400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4017070 con 0x7f3f041034a0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4005020 con 0x7f3f041034a0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0419a370 con 0x7f3f041034a0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400fa10 con 0x7f3f041034a0 2024-04-26T18:28:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.119+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f04075650 con 0x7f3f041034a0 2024-04-26T18:28:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.120+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0406a110 con 0x7f3f041034a0 2024-04-26T18:28:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.121+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ef4004a30 con 0x7f3f041034a0 2024-04-26T18:28:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.122+0000 7f3f00ff9700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 0x7f3eec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.122+0000 7f3f02ffd700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 0x7f3eec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:54.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.122+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3ef40d0a30 con 0x7f3f041034a0 2024-04-26T18:28:54.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.123+0000 7f3f02ffd700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 0x7f3eec0a62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800b5c0 tx=0x7f3ef8005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:54.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.126+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ef4020020 con 0x7f3f041034a0 2024-04-26T18:28:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.386+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f0402e490 con 0x7f3f041034a0 2024-04-26T18:28:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.387+0000 7f3f00ff9700 1 -- 172.21.15.37:0/353541587 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11884 (secure 0 0 0) 0x7f3ef409d0e0 con 0x7f3f041034a0 2024-04-26T18:28:54.387 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:28:54.387 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:28:50.545932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:28:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 msgr2=0x7f3eec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 0x7f3eec0a62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800b5c0 tx=0x7f3ef8005960 comp rx=0 tx=0).stop 2024-04-26T18:28:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 msgr2=0x7f3f0419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400d980 tx=0x7f3ef400dc90 comp rx=0 tx=0).stop 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 shutdown_connections 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f041094e0 0x7f3f04199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f041034a0 0x7f3f0419f760 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3eec0a3e30 0x7f3eec0a62f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 --2- 172.21.15.37:0/353541587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f04103e60 0x7f3f0419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 >> 172.21.15.37:0/353541587 conn(0x7f3f0407a590 msgr2=0x7f3f041022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 shutdown_connections 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:54.389+0000 7f3f09afe700 1 -- 172.21.15.37:0/353541587 wait complete. 2024-04-26T18:28:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-04-26T18:28:54.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:54 smithi037 ceph-mon[29171]: pgmap v140: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-26T18:28:54.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:54 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/353541587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:55.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:54 smithi120 ceph-mon[32244]: pgmap v140: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-26T18:28:55.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:54 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/353541587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:55.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:54 smithi170 ceph-mon[32697]: pgmap v140: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-26T18:28:55.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:54 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/353541587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:55.137 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id 4c5b6836 --release --await' 2024-04-26T18:28:55.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db --set-id 4c5b6836 --release --await 2024-04-26T18:28:55.386 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 -- 172.21.15.37:0/154089239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103e30 msgr2=0x7f05f0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 --2- 172.21.15.37:0/154089239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103e30 0x7f05f0110290 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b0a0 tx=0x7f05e800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 -- 172.21.15.37:0/154089239 shutdown_connections 2024-04-26T18:28:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 --2- 172.21.15.37:0/154089239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f0103e30 0x7f05f0110290 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 --2- 172.21.15.37:0/154089239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f01094c0 0x7f05f01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 --2- 172.21.15.37:0/154089239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 0x7f05f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 -- 172.21.15.37:0/154089239 >> 172.21.15.37:0/154089239 conn(0x7f05f007a580 msgr2=0x7f05f007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 -- 172.21.15.37:0/154089239 shutdown_connections 2024-04-26T18:28:57.074 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:57 smithi037 ceph-mon[29171]: pgmap v141: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.073+0000 7f05f6d15700 1 -- 172.21.15.37:0/154089239 wait complete. 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 Processor -- start 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 -- start start 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 0x7f05f01a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f0103e30 0x7f05f01a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01135c0 con 0x7f05f01094c0 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0113440 con 0x7f05f0103470 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f6d15700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0113740 con 0x7f05f0103e30 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f52b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f52b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55260/0 (socket says 172.21.15.37:55260) 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f52b2700 1 -- 172.21.15.37:0/699601128 learned_addr learned my addr 172.21.15.37:0/699601128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 0x7f05f01a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05effff700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f0103e30 0x7f05f01a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.074+0000 7f05f52b2700 1 -- 172.21.15.37:0/699601128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f0103e30 msgr2=0x7f05f01a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f52b2700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f0103e30 0x7f05f01a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f52b2700 1 -- 172.21.15.37:0/699601128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 msgr2=0x7f05f01a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f52b2700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 0x7f05f01a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f52b2700 1 -- 172.21.15.37:0/699601128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f010fb30 con 0x7f05f01094c0 2024-04-26T18:28:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f52b2700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b830 tx=0x7f05e8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05edffb700 1 -- 172.21.15.37:0/699601128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8021070 con 0x7f05f01094c0 2024-04-26T18:28:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f010fdc0 con 0x7f05f01094c0 2024-04-26T18:28:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05edffb700 1 -- 172.21.15.37:0/699601128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e8007e30 con 0x7f05f01094c0 2024-04-26T18:28:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05edffb700 1 -- 172.21.15.37:0/699601128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80047d0 con 0x7f05f01094c0 2024-04-26T18:28:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.075+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01acb90 con 0x7f05f01094c0 2024-04-26T18:28:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f05edffb700 1 -- 172.21.15.37:0/699601128 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05e8007980 con 0x7f05f01094c0 2024-04-26T18:28:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f05edffb700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 0x7f05e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f05edffb700 1 -- 172.21.15.37:0/699601128 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e80d18a0 con 0x7f05f01094c0 2024-04-26T18:28:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f06015a7b80 1 Processor -- start 2024-04-26T18:28:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 0x7f05e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.079+0000 7f06015a7b80 1 -- start start 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc592280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc58ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567bc4de930 con 0x5567bc5927c0 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567bc444a30 con 0x5567bc58c870 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f06015a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5567bc4da8f0 con 0x5567bc591ea0 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 0x7f05e00a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05e400abd0 tx=0x7f05e400a380 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55268/0 (socket says 172.21.15.37:55268) 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 -- 172.21.15.37:0/944194581 learned_addr learned my addr 172.21.15.37:0/944194581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc592280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05f52b2700 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc58ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 -- 172.21.15.37:0/944194581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 msgr2=0x5567bc592280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc592280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 -- 172.21.15.37:0/944194581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 msgr2=0x5567bc58ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc58ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.080+0000 7f05effff700 1 -- 172.21.15.37:0/944194581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567bc42b9d0 con 0x5567bc5927c0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f05effff700 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00bce0 tx=0x7f05dc00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b05570c2b1eae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f05daffd700 1 -- 172.21.15.37:0/944194581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc015070 con 0x5567bc5927c0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f05daffd700 1 -- 172.21.15.37:0/944194581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc00fbe0 con 0x5567bc5927c0 2024-04-26T18:28:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f05daffd700 1 -- 172.21.15.37:0/944194581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0116a0 con 0x5567bc5927c0 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f06015a7b80 1 -- 172.21.15.37:0/944194581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 msgr2=0x5567bc58c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f06015a7b80 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00bce0 tx=0x7f05dc00f360 comp rx=0 tx=0).stop 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f06015a7b80 1 -- 172.21.15.37:0/944194581 shutdown_connections 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.081+0000 7f06015a7b80 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc58ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc592280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 --2- 172.21.15.37:0/944194581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc58c250 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 -- 172.21.15.37:0/944194581 >> 172.21.15.37:0/944194581 conn(0x5567bc4ba6c0 msgr2=0x5567bc587000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 -- 172.21.15.37:0/944194581 shutdown_connections 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 -- 172.21.15.37:0/944194581 wait complete. 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 -- start start 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 Processor -- start 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc598490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc595830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.082+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f06015a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567bc599230 con 0x5567bc5927c0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f06015a7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5567bc498b80 con 0x5567bc591ea0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f06015a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5567bc5993b0 con 0x5567bc58c870 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55282/0 (socket says 172.21.15.37:55282) 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 -- 172.21.15.37:0/1794660848 learned_addr learned my addr 172.21.15.37:0/1794660848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05effff700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc595830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc598490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f06015a7b80 10 client.0 _ll_register_callbacks cb 0x5567bc476a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 -- 172.21.15.37:0/1794660848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 msgr2=0x5567bc598490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc598490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 -- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 msgr2=0x5567bc595830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc595830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 -- 172.21.15.37:0/1794660848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bc49c580 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05effff700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc595830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05dc00fbe0 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05f52b2700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f05e809a290 tx=0x7f05e80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e3f0c8c41c8649 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05cffff700 10 client.15828 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05cffff700 1 -- 172.21.15.37:0/1794660848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e802a030 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.083+0000 7f05cffff700 1 -- 172.21.15.37:0/1794660848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e80a2e00 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f05cffff700 1 -- 172.21.15.37:0/1794660848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05e809d8c0 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 10 client.15828 fetch_fsmap learned FSMap version 28 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5567bc5a3950 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f05cffff700 1 -- 172.21.15.37:0/1794660848 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f05e8011070 con 0x5567bc5927c0 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 10 client.15828 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:28:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 20 client.15828 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5a5e20 0x5567bc5a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 4 client.15828 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567bc4ec310 con 0x5567bc5a5e20 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.084+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5a5e20 0x5567bc5a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.085+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5a5e20 0x5567bc5a8210 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18118a52eedacf55 server_cookie=b5a5944cbcc30cc6 in_seq=0 out_seq=0 2024-04-26T18:28:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.085+0000 7f05cffff700 10 client.15828 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.085+0000 7f05cffff700 1 -- 172.21.15.37:0/1794660848 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567bc4ec310 con 0x5567bc5a5e20 2024-04-26T18:28:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.085+0000 7f05cffff700 10 client.15828 handle_command_reply: tid=0 2024-04-26T18:28:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 client.15828 shutdown 2024-04-26T18:28:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 msgr2=0x5567bc595d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f05e809a290 tx=0x7f05e80099e0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 shutdown_connections 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5a5e20 0x5567bc5a8210 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc5927c0 0x5567bc595d70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc598490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 --2- 172.21.15.37:0/1794660848 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc595830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 >> 172.21.15.37:0/1794660848 conn(0x5567bc4ba6c0 msgr2=0x5567bc58b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f05cffff700 0 client.15828 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 shutdown_connections 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 1 -- 172.21.15.37:0/1794660848 wait complete. 2024-04-26T18:28:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.086+0000 7f06015a7b80 20 client.15828 trim_cache size 0 max 16384 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.105+0000 7f06015a7b80 1 Processor -- start 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.105+0000 7f06015a7b80 1 -- start start 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc593370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc5a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b6ad0 con 0x5567bc4ba6c0 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b4360 con 0x5567bc591ea0 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f06015a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b21b0 con 0x5567bc58c870 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55292/0 (socket says 172.21.15.37:55292) 2024-04-26T18:28:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 -- 172.21.15.37:0/473093508 learned_addr learned my addr 172.21.15.37:0/473093508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05effff700 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc593370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f52b2700 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc5a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 -- 172.21.15.37:0/473093508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 msgr2=0x5567bc593370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc593370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 -- 172.21.15.37:0/473093508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 msgr2=0x5567bc5a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc5a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 -- 172.21.15.37:0/473093508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567bc587cd0 con 0x5567bc4ba6c0 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f05e400f750 tx=0x7f05e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ed0ef919331c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05da7fc700 1 -- 172.21.15.37:0/473093508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4016070 con 0x5567bc4ba6c0 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05da7fc700 1 -- 172.21.15.37:0/473093508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e402e070 con 0x5567bc4ba6c0 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.106+0000 7f05da7fc700 1 -- 172.21.15.37:0/473093508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4010720 con 0x5567bc4ba6c0 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- 172.21.15.37:0/473093508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 msgr2=0x5567bc58c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f05e400f750 tx=0x7f05e4012000 comp rx=0 tx=0).stop 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- 172.21.15.37:0/473093508 shutdown_connections 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc591ea0 0x5567bc5a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc58c250 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 --2- 172.21.15.37:0/473093508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc58c870 0x5567bc593370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- 172.21.15.37:0/473093508 >> 172.21.15.37:0/473093508 conn(0x5567bc587330 msgr2=0x5567bc592c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- 172.21.15.37:0/473093508 shutdown_connections 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- 172.21.15.37:0/473093508 wait complete. 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.107+0000 7f06015a7b80 1 -- start start 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 Processor -- start 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc59bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc60c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b2820 con 0x5567bc4ba6c0 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b3c90 con 0x5567bc58c870 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b2ef0 con 0x5567bc591ea0 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f06015a7b80 10 client.0 _ll_register_callbacks cb 0x5567bc476a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55300/0 (socket says 172.21.15.37:55300) 2024-04-26T18:28:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 -- 172.21.15.37:0/2248369771 learned_addr learned my addr 172.21.15.37:0/2248369771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f52b2700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc60c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05effff700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc59bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 -- 172.21.15.37:0/2248369771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 msgr2=0x5567bc60c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc60c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 -- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 msgr2=0x5567bc59bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc59bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.108+0000 7f05f4ab1700 1 -- 172.21.15.37:0/2248369771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bc60c910 con 0x5567bc4ba6c0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05f4ab1700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f05e40297e0 tx=0x7f05e40298c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe200cc2dbcf502c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4016070 con 0x5567bc4ba6c0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f05e4003cf0 con 0x5567bc4ba6c0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e402e070 con 0x5567bc4ba6c0 2024-04-26T18:28:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05e4042050 con 0x5567bc4ba6c0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 10 client.15834 fetch_fsmap learned FSMap version 28 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567bc60dc50 con 0x5567bc4ba6c0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e402e070 con 0x5567bc4ba6c0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f05e4019040 con 0x5567bc4ba6c0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 10 client.15834 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 20 client.15834 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.109+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5927c0 0x5567bc610340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.110+0000 7f06015a7b80 4 client.15834 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4c5b6836", "release": true, "await": true}] 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.110+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4c5b6836", "release": true, "await": true}) -- 0x5567bc4e63b0 con 0x5567bc5927c0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.110+0000 7f05effff700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5927c0 0x5567bc610340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.110+0000 7f05effff700 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5927c0 0x5567bc610340 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c509cfacfa1ee99c server_cookie=e828fbe414c23c2e in_seq=0 out_seq=0 2024-04-26T18:28:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.110+0000 7f05cffff700 10 client.15834 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.118+0000 7f05cffff700 1 -- 172.21.15.37:0/2248369771 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5567bc4e63b0 con 0x5567bc5927c0 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.118+0000 7f05cffff700 10 client.15834 handle_command_reply: tid=0 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.118+0000 7f06015a7b80 1 client.15834 shutdown 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 msgr2=0x5567bc59b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f05e40297e0 tx=0x7f05e40298c0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 shutdown_connections 2024-04-26T18:28:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5567bc5927c0 0x5567bc610340 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5567bc591ea0 0x5567bc60c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567bc4ba6c0 0x5567bc59b810 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f05cffff700 0 client.15834 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 --2- 172.21.15.37:0/2248369771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5567bc58c870 0x5567bc59bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 >> 172.21.15.37:0/2248369771 conn(0x5567bc587330 msgr2=0x5567bc580670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 shutdown_connections 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 1 -- 172.21.15.37:0/2248369771 wait complete. 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f06015a7b80 20 client.15834 trim_cache size 0 max 16384 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:28:57.120 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 28, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "4c5b6836": { 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 79.5, 2024-04-26T18:28:57.121 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:28:57.122 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 msgr2=0x7f05e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.119+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 0x7f05e00a6270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f05e400abd0 tx=0x7f05e400a380 comp rx=0 tx=0).stop 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 msgr2=0x7f05f010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b830 tx=0x7f05e8004ce0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 shutdown_connections 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05f01094c0 0x7f05f010f3a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f05e00a3db0 0x7f05e00a6270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f05f0103470 0x7f05f01a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 --2- 172.21.15.37:0/699601128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05f0103e30 0x7f05f01a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 >> 172.21.15.37:0/699601128 conn(0x7f05f007a580 msgr2=0x7f05f0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 shutdown_connections 2024-04-26T18:28:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:57.120+0000 7f05f6d15700 1 -- 172.21.15.37:0/699601128 wait complete. 2024-04-26T18:28:57.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:57 smithi170 ceph-mon[32697]: pgmap v141: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:28:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:57 smithi120 ceph-mon[32244]: pgmap v141: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:28:57.839 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4c5b6836', seconds elapsed: 34.6 2024-04-26T18:28:57.840 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:28:57.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 -- 172.21.15.37:0/88222578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38101860 msgr2=0x7fcd38101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/88222578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38101860 0x7fcd38101ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009930 tx=0x7fcd2002a420 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 -- 172.21.15.37:0/88222578 shutdown_connections 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/88222578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38102220 0x7fcd3810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/88222578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38101860 0x7fcd38101ce0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/88222578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd38107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 -- 172.21.15.37:0/88222578 >> 172.21.15.37:0/88222578 conn(0x7fcd380789b0 msgr2=0x7fcd38078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 -- 172.21.15.37:0/88222578 shutdown_connections 2024-04-26T18:28:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.730+0000 7fcd3cc72640 1 -- 172.21.15.37:0/88222578 wait complete. 2024-04-26T18:28:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.731+0000 7fcd3cc72640 1 Processor -- start 2024-04-26T18:28:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.731+0000 7fcd3cc72640 1 -- start start 2024-04-26T18:28:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38101860 0x7fcd38198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd3819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38111ae0 con 0x7fcd38102220 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38111960 con 0x7fcd38107820 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3cc72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38111c60 con 0x7fcd38101860 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55310/0 (socket says 172.21.15.37:55310) 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/4168125150 learned_addr learned my addr 172.21.15.37:0/4168125150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd3819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd3659c640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38101860 0x7fcd38198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 -- 172.21.15.37:0/4168125150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38101860 msgr2=0x7fcd38198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38101860 0x7fcd38198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 -- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 msgr2=0x7fcd38198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd36d9d640 1 -- 172.21.15.37:0/4168125150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3819d030 con 0x7fcd38107820 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.732+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd3819c900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000e940 tx=0x7fcd3000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd2ffff640 1 -- 172.21.15.37:0/4168125150 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004d20 con 0x7fcd38107820 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3819d320 con 0x7fcd38107820 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd2ffff640 1 -- 172.21.15.37:0/4168125150 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd30005020 con 0x7fcd38107820 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd2ffff640 1 -- 172.21.15.37:0/4168125150 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300116d0 con 0x7fcd38107820 2024-04-26T18:28:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.733+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd38069fe0 con 0x7fcd38107820 2024-04-26T18:28:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.735+0000 7fcd2ffff640 1 -- 172.21.15.37:0/4168125150 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd30005460 con 0x7fcd38107820 2024-04-26T18:28:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.735+0000 7fcd3f98f740 1 Processor -- start 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 -- start start 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c6740 0x5566580d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55665802c460 con 0x5566580c5e20 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556657fe6590 con 0x5566580c6740 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556658028420 con 0x5566580d54f0 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55322/0 (socket says 172.21.15.37:55322) 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/1458917152 learned_addr learned my addr 172.21.15.37:0/1458917152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2ffff640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 0x7fcd240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd3659c640 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c6740 0x5566580d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2ffff640 1 -- 172.21.15.37:0/4168125150 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fcd300d0b90 con 0x7fcd38107820 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/1458917152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 msgr2=0x5566580d78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 0x7fcd240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/1458917152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c6740 msgr2=0x5566580d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c6740 0x5566580d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.736+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/1458917152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556657ff30d0 con 0x5566580c5e20 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.737+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd20032fd0 tx=0x7fcd200058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee24b5db9fe5849 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.737+0000 7fcd36d9d640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 0x7fcd240a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd300995a0 tx=0x7fcd30005750 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.737+0000 7fcd2cd99640 1 -- 172.21.15.37:0/1458917152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20035040 con 0x5566580c5e20 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.737+0000 7fcd2cd99640 1 -- 172.21.15.37:0/1458917152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd200054d0 con 0x5566580c5e20 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.737+0000 7fcd2cd99640 1 -- 172.21.15.37:0/1458917152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200049e0 con 0x5566580c5e20 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1458917152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 msgr2=0x5566580c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd20032fd0 tx=0x7fcd200058d0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1458917152 shutdown_connections 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c6740 0x5566580d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1458917152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c5e20 0x5566580c6200 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1458917152 >> 172.21.15.37:0/1458917152 conn(0x5566580c2410 msgr2=0x5566580c27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1458917152 shutdown_connections 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1458917152 wait complete. 2024-04-26T18:28:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- start start 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 Processor -- start 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.738+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566580d80c0 con 0x5566580c6740 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556657fb3b60 con 0x5566580c5e20 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5566580d8240 con 0x5566580d54f0 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3f98f740 10 client.0 _ll_register_callbacks cb 0x556657fd82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56064/0 (socket says 172.21.15.37:56064) 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55328/0 (socket says 172.21.15.37:55328) 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/3476207577 learned_addr learned my addr 172.21.15.37:0/3476207577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 -- 172.21.15.37:0/3476207577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 msgr2=0x5566580d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 -- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 msgr2=0x5566580d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 -- 172.21.15.37:0/3476207577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556657d3e210 con 0x5566580c6740 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3659c640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd280126d0 tx=0x7fcd28012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e87bdf8e97d436 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28004030 con 0x5566580c6740 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fcd280041d0 con 0x5566580c6740 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.739+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd200054d0 con 0x5566580c6740 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd280043d0 con 0x5566580c6740 2024-04-26T18:28:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556657e0a620 con 0x5566580c6740 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd200054d0 con 0x5566580c6740 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fcd280039e0 con 0x5566580c6740 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 20 client.15846 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x5566580e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 4 client.15846 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55665803a070 con 0x5566580e56a0 2024-04-26T18:28:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.740+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x5566580e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.741+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x5566580e7b00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897166734236cec4 server_cookie=a754cbf1c2e24123 in_seq=0 out_seq=0 2024-04-26T18:28:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.741+0000 7fcd1dffb640 10 client.15846 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd1dffb640 1 -- 172.21.15.37:0/3476207577 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55665803a070 con 0x5566580e56a0 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd1dffb640 10 client.15846 handle_command_reply: tid=0 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 client.15846 shutdown 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 msgr2=0x5566580d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd280126d0 tx=0x7fcd28012b90 comp rx=0 tx=0).stop 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 shutdown_connections 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x5566580e7b00 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd1dffb640 0 client.15846 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d29f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/3476207577 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 >> 172.21.15.37:0/3476207577 conn(0x5566580c2410 msgr2=0x5566580c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 shutdown_connections 2024-04-26T18:28:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 1 -- 172.21.15.37:0/3476207577 wait complete. 2024-04-26T18:28:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.742+0000 7fcd3f98f740 20 client.15846 trim_cache size 0 max 16384 2024-04-26T18:28:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 Processor -- start 2024-04-26T18:28:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 -- start start 2024-04-26T18:28:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381aec80 con 0x5566580c6740 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ac550 con 0x5566580c5e20 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381aa3a0 con 0x5566580d54f0 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55342/0 (socket says 172.21.15.37:55342) 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 -- 172.21.15.37:0/4018454743 learned_addr learned my addr 172.21.15.37:0/4018454743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 -- 172.21.15.37:0/4018454743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 msgr2=0x5566580ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580ed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 -- 172.21.15.37:0/4018454743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 msgr2=0x5566580e0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.759+0000 7fcd3659c640 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580e0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.760+0000 7fcd3659c640 1 -- 172.21.15.37:0/4018454743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566580e4210 con 0x5566580c6740 2024-04-26T18:28:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.760+0000 7fcd3659c640 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000c00 tx=0x7fcd280057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05ae09c1b5345a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.760+0000 7fcd1ffff640 1 -- 172.21.15.37:0/4018454743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28028030 con 0x5566580c6740 2024-04-26T18:28:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.760+0000 7fcd1ffff640 1 -- 172.21.15.37:0/4018454743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2801cdb0 con 0x5566580c6740 2024-04-26T18:28:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.760+0000 7fcd1ffff640 1 -- 172.21.15.37:0/4018454743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2801c570 con 0x5566580c6740 2024-04-26T18:28:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- 172.21.15.37:0/4018454743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 msgr2=0x5566580d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000c00 tx=0x7fcd280057d0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- 172.21.15.37:0/4018454743 shutdown_connections 2024-04-26T18:28:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580d54f0 0x5566580ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580c6740 0x5566580d43f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/4018454743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580e0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- 172.21.15.37:0/4018454743 >> 172.21.15.37:0/4018454743 conn(0x5566580c2410 msgr2=0x5566580d3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- 172.21.15.37:0/4018454743 shutdown_connections 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- 172.21.15.37:0/4018454743 wait complete. 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 -- start start 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.761+0000 7fcd3f98f740 1 Processor -- start 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 0x5566580d9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580d54f0 0x5566580d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381aaa10 con 0x5566580d54f0 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381abe80 con 0x5566580c5e20 2024-04-26T18:28:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ab0e0 con 0x5566580c6740 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3f98f740 10 client.0 _ll_register_callbacks cb 0x7fcd28009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd2dd9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56074/0 (socket says 172.21.15.37:56074) 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 0x5566580d9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd2dd9b640 1 -- 172.21.15.37:0/1071628496 learned_addr learned my addr 172.21.15.37:0/1071628496 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580d54f0 0x5566580d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 -- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 msgr2=0x5566580d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 -- 172.21.15.37:0/1071628496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580d54f0 msgr2=0x5566580d9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580d54f0 0x5566580d9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 -- 172.21.15.37:0/1071628496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665814af20 con 0x5566580c6740 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd3659c640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 0x5566580d9690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd28012bd0 tx=0x7fcd28004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c819810fe46ddc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.762+0000 7fcd1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2801cac0 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2801cdb0 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fcd28005050 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd2803a050 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd3f98f740 10 client.25426 fetch_fsmap learned FSMap version 28 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55665814c230 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.763+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2801cdb0 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fcd28035030 con 0x5566580c6740 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 10 client.25426 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 20 client.25426 populate_metadata read hostname 'smithi037' 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x55665814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 4 client.25426 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556658033ee0 con 0x5566580e56a0 2024-04-26T18:28:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x55665814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd1dffb640 10 client.25426 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.764+0000 7fcd2dd9b640 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x55665814e7e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692340046df482e4 server_cookie=ead80b3c7036f804 in_seq=0 out_seq=0 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd1dffb640 1 -- 172.21.15.37:0/1071628496 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556658033ee0 con 0x5566580e56a0 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd1dffb640 10 client.25426 handle_command_reply: tid=0 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 client.25426 shutdown 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 msgr2=0x5566580d9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 0x5566580d9690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd28012bd0 tx=0x7fcd28004360 comp rx=0 tx=0).stop 2024-04-26T18:28:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 shutdown_connections 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5566580e56a0 0x55665814e7e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566580d54f0 0x5566580d9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5566580c6740 0x5566580d9690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 --2- 172.21.15.37:0/1071628496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5566580c5e20 0x5566580d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd1dffb640 0 client.25426 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.777+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 >> 172.21.15.37:0/1071628496 conn(0x5566580c2410 msgr2=0x5566580d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 shutdown_connections 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3f98f740 1 -- 172.21.15.37:0/1071628496 wait complete. 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3f98f740 20 client.25426 trim_cache size 0 max 16384 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:28:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:28:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:28:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:28:58.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 msgr2=0x7fcd240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 0x7fcd240a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd300995a0 tx=0x7fcd30005750 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 msgr2=0x7fcd3819c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd3819c900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000e940 tx=0x7fcd3000ee00 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 shutdown_connections 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fcd240a3dc0 0x7fcd240a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcd38107820 0x7fcd3819c900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd38102220 0x7fcd38198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 --2- 172.21.15.37:0/4168125150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd38101860 0x7fcd38198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 >> 172.21.15.37:0/4168125150 conn(0x7fcd380789b0 msgr2=0x7fcd38104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 shutdown_connections 2024-04-26T18:28:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:28:58.778+0000 7fcd3cc72640 1 -- 172.21.15.37:0/4168125150 wait complete. 2024-04-26T18:28:59.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:28:59 smithi170 ceph-mon[32697]: pgmap v142: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:28:59.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:28:59 smithi037 ceph-mon[29171]: pgmap v142: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:28:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:28:59 smithi120 ceph-mon[32244]: pgmap v142: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:28:59.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.997+0000 7f44f0897700 1 -- 172.21.15.37:0/1143102462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec06bad0 msgr2=0x7f44ec1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:59.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.997+0000 7f44f0897700 1 --2- 172.21.15.37:0/1143102462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec06bad0 0x7f44ec1102e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b0a0 tx=0x7f44e400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:59.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 -- 172.21.15.37:0/1143102462 shutdown_connections 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 --2- 172.21.15.37:0/1143102462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec06bad0 0x7f44ec1102e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 --2- 172.21.15.37:0/1143102462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec1094a0 0x7f44ec109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 --2- 172.21.15.37:0/1143102462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 -- 172.21.15.37:0/1143102462 >> 172.21.15.37:0/1143102462 conn(0x7f44ec0792d0 msgr2=0x7f44ec0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 -- 172.21.15.37:0/1143102462 shutdown_connections 2024-04-26T18:28:59.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 -- 172.21.15.37:0/1143102462 wait complete. 2024-04-26T18:29:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 Processor -- start 2024-04-26T18:29:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.998+0000 7f44f0897700 1 -- start start 2024-04-26T18:29:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec06bad0 0x7f44ec19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1135c0 con 0x7f44ec1094a0 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec113440 con 0x7f44ec06b130 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44f0897700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec113740 con 0x7f44ec06bad0 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ea59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ea59c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56092/0 (socket says 172.21.15.37:56092) 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55372/0 (socket says 172.21.15.37:55372) 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ea59c700 1 -- 172.21.15.37:0/3843538712 learned_addr learned my addr 172.21.15.37:0/3843538712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44e9d9b700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec06bad0 0x7f44ec19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 -- 172.21.15.37:0/3843538712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec06bad0 msgr2=0x7f44ec19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec06bad0 0x7f44ec19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 -- 172.21.15.37:0/3843538712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 msgr2=0x7f44ec19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ead9d700 1 -- 172.21.15.37:0/3843538712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec19a0a0 con 0x7f44ec1094a0 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:28:59.999+0000 7f44ea59c700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44ead9d700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b070 tx=0x7f44e4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4021070 con 0x7f44ec1094a0 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec19a330 con 0x7f44ec1094a0 2024-04-26T18:29:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e4007e30 con 0x7f44ec1094a0 2024-04-26T18:29:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40045d0 con 0x7f44ec1094a0 2024-04-26T18:29:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.000+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec1a65e0 con 0x7f44ec1094a0 2024-04-26T18:29:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.001+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec06a110 con 0x7f44ec1094a0 2024-04-26T18:29:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.002+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44e4003c00 con 0x7f44ec1094a0 2024-04-26T18:29:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.003+0000 7f44db7fe700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 0x7f44d40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.003+0000 7f44ea59c700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 0x7f44d40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.003+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f44e400f020 con 0x7f44ec1094a0 2024-04-26T18:29:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.004+0000 7f44ea59c700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 0x7f44d40a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44ec19b0c0 tx=0x7f44dc009260 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:00.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.007+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44e402a020 con 0x7f44ec1094a0 2024-04-26T18:29:00.072 INFO:tasks.workunit.client.0.smithi037.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-26T18:29:00.072 INFO:tasks.workunit.client.0.smithi037.stdout:All 50000 operations completed A-OK! 2024-04-26T18:29:00.075 INFO:tasks.workunit.client.0.smithi037.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-26T18:29:00.077 INFO:tasks.workunit.client.0.smithi037.stdout:Seed set to 1 2024-04-26T18:29:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.274+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44ec02e490 con 0x7f44ec1094a0 2024-04-26T18:29:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.275+0000 7f44db7fe700 1 -- 172.21.15.37:0/3843538712 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11884 (secure 0 0 0) 0x7f44e409e5e0 con 0x7f44ec1094a0 2024-04-26T18:29:00.276 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:29:00.276 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:28:50.545932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 msgr2=0x7f44d40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 0x7f44d40a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44ec19b0c0 tx=0x7f44dc009260 comp rx=0 tx=0).stop 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 msgr2=0x7f44ec199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b070 tx=0x7f44e4004d00 comp rx=0 tx=0).stop 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 shutdown_connections 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44ec1094a0 0x7f44ec199920 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44d40a3db0 0x7f44d40a6270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44ec06b130 0x7f44ec19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 --2- 172.21.15.37:0/3843538712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44ec06bad0 0x7f44ec19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.277+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 >> 172.21.15.37:0/3843538712 conn(0x7f44ec0792d0 msgr2=0x7f44ec100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.278+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 shutdown_connections 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:00.278+0000 7f44f0897700 1 -- 172.21.15.37:0/3843538712 wait complete. 2024-04-26T18:29:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-04-26T18:29:00.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:29:00.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:29:01.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:01 smithi037 ceph-mon[29171]: pgmap v143: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:29:01.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:01 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3843538712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:01.234 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:01.293 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T18:29:01.293 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T18:29:01.297 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T18:29:01.297 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T18:29:01.298 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T18:29:01.302 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T18:29:01.305 INFO:tasks.workunit.client.0.smithi037.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T18:29:01.305 INFO:tasks.workunit.client.0.smithi037.stdout:skipping zero size read 2024-04-26T18:29:01.316 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x1c37b90 2024-04-26T18:29:01.323 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x5bd330d 2024-04-26T18:29:01.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:01 smithi170 ceph-mon[32697]: pgmap v143: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:29:01.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:01 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3843538712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:01.344 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x637bf99 2024-04-26T18:29:01.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:01 smithi120 ceph-mon[32244]: pgmap v143: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-04-26T18:29:01.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:01 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3843538712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:01.524 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x63d9b9c 2024-04-26T18:29:03.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 -- 172.21.15.37:0/1356187066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 msgr2=0x7ff0b8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 --2- 172.21.15.37:0/1356187066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b8102960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00099c0 tx=0x7ff0b0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 -- 172.21.15.37:0/1356187066 shutdown_connections 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 --2- 172.21.15.37:0/1356187066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 --2- 172.21.15.37:0/1356187066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b8102960 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 --2- 172.21.15.37:0/1356187066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 0x7ff0b8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 -- 172.21.15.37:0/1356187066 >> 172.21.15.37:0/1356187066 conn(0x7ff0b80792c0 msgr2=0x7ff0b80796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 -- 172.21.15.37:0/1356187066 shutdown_connections 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.079+0000 7ff0be244700 1 -- 172.21.15.37:0/1356187066 wait complete. 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 Processor -- start 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 -- start start 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b81092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 0x7ff0b8109800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8112550 con 0x7ff0b8102580 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81123d0 con 0x7ff0b8107650 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0be244700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81126d0 con 0x7ff0b8102f30 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56116/0 (socket says 172.21.15.37:56116) 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0b7fff700 1 -- 172.21.15.37:0/3591515088 learned_addr learned my addr 172.21.15.37:0/3591515088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 0x7ff0b8109800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.080+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b81092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 -- 172.21.15.37:0/3591515088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 msgr2=0x7ff0b8109800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 0x7ff0b8109800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 -- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 msgr2=0x7ff0b81092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b81092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 -- 172.21.15.37:0/3591515088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b81a8030 con 0x7ff0b8107650 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c370 tx=0x7ff0a800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b4ff9700 1 -- 172.21.15.37:0/3591515088 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80092e0 con 0x7ff0b8107650 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b4ff9700 1 -- 172.21.15.37:0/3591515088 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a8003a00 con 0x7ff0b8107650 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0b4ff9700 1 -- 172.21.15.37:0/3591515088 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80046f0 con 0x7ff0b8107650 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.081+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b81a82c0 con 0x7ff0b8107650 2024-04-26T18:29:03.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.082+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b81a87b0 con 0x7ff0b8107650 2024-04-26T18:29:03.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.084+0000 7ff0b4ff9700 1 -- 172.21.15.37:0/3591515088 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0a801a020 con 0x7ff0b8107650 2024-04-26T18:29:03.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.084+0000 7ff0b4ff9700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 0x7ff0a40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.085+0000 7ff0b4ff9700 1 -- 172.21.15.37:0/3591515088 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff0a80cff90 con 0x7ff0b8107650 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.085+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 0x7ff0a40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.085+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 0x7ff0a40a6390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0003760 tx=0x7ff0b0005fb0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 Processor -- start 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 -- start start 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b49610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a38a9c8d0 con 0x564a38b49230 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564a38a3b770 con 0x564a38b49b50 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a38a98890 con 0x564a38b54a00 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b49610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56122/0 (socket says 172.21.15.37:56122) 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b6ffd700 1 -- 172.21.15.37:0/2075721683 learned_addr learned my addr 172.21.15.37:0/2075721683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:40846/0 (socket says 172.21.15.37:40846) 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.086+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b49610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55402/0 (socket says 172.21.15.37:55402) 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2075721683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 msgr2=0x564a38b544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2075721683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 msgr2=0x564a38b49610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b49610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2075721683 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a389e9a60 con 0x564a38b54a00 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:03.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80cd6d0 tx=0x7ff0a809be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb21c9d70210be00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0adffb700 1 -- 172.21.15.37:0/2075721683 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80b5070 con 0x564a38b54a00 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0adffb700 1 -- 172.21.15.37:0/2075721683 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a809c620 con 0x564a38b54a00 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.087+0000 7ff0adffb700 1 -- 172.21.15.37:0/2075721683 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80192c0 con 0x564a38b54a00 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2075721683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 msgr2=0x564a38b56df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80cd6d0 tx=0x7ff0a809be20 comp rx=0 tx=0).stop 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2075721683 shutdown_connections 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b54a00 0x564a38b56df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b49610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2075721683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49b50 0x564a38b544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2075721683 >> 172.21.15.37:0/2075721683 conn(0x564a38a76ed0 msgr2=0x564a38b45720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2075721683 shutdown_connections 2024-04-26T18:29:03.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.088+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2075721683 wait complete. 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 -- start start 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 Processor -- start 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b54220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b51520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b51a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564a38b575d0 con 0x564a38b49230 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564a38a02ae0 con 0x564a38b54a00 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564a38b57750 con 0x564a38b49b50 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0c8ad6b80 10 client.0 _ll_register_callbacks cb 0x564a38a30710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b51a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b54220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0b6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b51520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b51a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56136/0 (socket says 172.21.15.37:56136) 2024-04-26T18:29:03.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.089+0000 7ff0b7fff700 1 -- 172.21.15.37:0/3252102869 learned_addr learned my addr 172.21.15.37:0/3252102869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 -- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 msgr2=0x564a38b51a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b51a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 -- 172.21.15.37:0/3252102869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 msgr2=0x564a38b54220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b54220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 -- 172.21.15.37:0/3252102869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a38b521b0 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b51520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0012770 tx=0x7ff0a0012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8235d88c31628d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0004050 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a809c620 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff0a00041f0 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a00043f0 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0c8ad6b80 10 client.25444 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.090+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a38b621a0 con 0x564a38b49b50 2024-04-26T18:29:03.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a809c620 con 0x564a38b49b50 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff0a0010630 con 0x564a38b49b50 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 10 client.25444 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 10 client.25444 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 20 client.25444 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38b64730 0x564a38b66b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 4 client.25444 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.091+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a38aaa2b0 con 0x564a38b64730 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.092+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38b64730 0x564a38b66b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.092+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38b64730 0x564a38b66b20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38812a146ddf64af server_cookie=f652e95e21665df9 in_seq=0 out_seq=0 2024-04-26T18:29:03.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.092+0000 7ff096ffd700 10 client.25444 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:03.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff096ffd700 1 -- 172.21.15.37:0/3252102869 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a38aaa2b0 con 0x564a38b64730 2024-04-26T18:29:03.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff096ffd700 10 client.25444 handle_command_reply: tid=0 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 client.25444 shutdown 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 msgr2=0x564a38b51520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b51520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0012770 tx=0x7ff0a0012a80 comp rx=0 tx=0).stop 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 shutdown_connections 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38b64730 0x564a38b66b20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b51a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b54220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/3252102869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b51520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 >> 172.21.15.37:0/3252102869 conn(0x564a38a76ed0 msgr2=0x564a38b45350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.093+0000 7ff096ffd700 0 client.25444 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.094+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 shutdown_connections 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.094+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/3252102869 wait complete. 2024-04-26T18:29:03.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.094+0000 7ff0c8ad6b80 20 client.25444 trim_cache size 0 max 16384 2024-04-26T18:29:03.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.116+0000 7ff0c8ad6b80 1 Processor -- start 2024-04-26T18:29:03.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.116+0000 7ff0c8ad6b80 1 -- start start 2024-04-26T18:29:03.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b4caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81b2890 con 0x564a38b54a00 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81b0120 con 0x564a38b49230 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81adf70 con 0x564a38b49b50 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55424/0 (socket says 172.21.15.37:55424) 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 -- 172.21.15.37:0/1210455850 learned_addr learned my addr 172.21.15.37:0/1210455850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:56144/0 (socket says 172.21.15.37:56144) 2024-04-26T18:29:03.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b4caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 -- 172.21.15.37:0/1210455850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 msgr2=0x564a38b4caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b4caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 -- 172.21.15.37:0/1210455850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 msgr2=0x564a38b458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b7fff700 1 -- 172.21.15.37:0/1210455850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a38b52420 con 0x564a38b54a00 2024-04-26T18:29:03.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b4caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.117+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.118+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff0a809bc20 tx=0x7ff0a809bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb945d7504f3dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.118+0000 7ff0ad7fa700 1 -- 172.21.15.37:0/1210455850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80b5040 con 0x564a38b54a00 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.118+0000 7ff0ad7fa700 1 -- 172.21.15.37:0/1210455850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a80a02d0 con 0x564a38b54a00 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.118+0000 7ff0ad7fa700 1 -- 172.21.15.37:0/1210455850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a809c180 con 0x564a38b54a00 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/1210455850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 msgr2=0x564a38b6c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff0a809bc20 tx=0x7ff0a809bb70 comp rx=0 tx=0).stop 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/1210455850 shutdown_connections 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b54a00 0x564a38b6c460 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b49230 0x564a38b458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/1210455850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b4caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/1210455850 >> 172.21.15.37:0/1210455850 conn(0x564a38a76ed0 msgr2=0x564a38b4c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:03.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/1210455850 shutdown_connections 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/1210455850 wait complete. 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.119+0000 7ff0c8ad6b80 1 -- start start 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 Processor -- start 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b58a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81ae5e0 con 0x564a38b49230 2024-04-26T18:29:03.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81afa50 con 0x564a38b54a00 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81aecb0 con 0x564a38b49b50 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0c8ad6b80 10 client.0 _ll_register_callbacks cb 0x564a38a30710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55426/0 (socket says 172.21.15.37:55426) 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b77fe700 1 -- 172.21.15.37:0/2117546844 learned_addr learned my addr 172.21.15.37:0/2117546844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b6ffd700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b58a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2117546844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 msgr2=0x564a38b58a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.120+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b58a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2117546844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 msgr2=0x564a38b58550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0b7fff700 1 -- 172.21.15.37:0/2117546844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a38bc9e90 con 0x564a38b54a00 2024-04-26T18:29:03.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0b7fff700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b58fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c340 tx=0x7ff0a8098810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e08bb4fe660d80e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a80a4b90 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff0a80850e0 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a80a02d0 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a809c180 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0c8ad6b80 10 client.25499 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a38bcb110 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.121+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a80a02d0 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff0a809f030 con 0x564a38b54a00 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 10 client.25499 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 10 client.25499 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:29:03.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 20 client.25499 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38bcb340 0x564a38bcd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 4 client.25499 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x564a38aa4350 con 0x564a38bcb340 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38bcb340 0x564a38bcd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff096ffd700 10 client.25499 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:03.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:03.122+0000 7ff0b77fe700 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38bcb340 0x564a38bcd730 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925298758fe69419 server_cookie=a156d58912618749 in_seq=0 out_seq=0 2024-04-26T18:29:03.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:03 smithi037 ceph-mon[29171]: pgmap v144: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 KiB/s wr, 151 op/s 2024-04-26T18:29:03.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:03 smithi170 ceph-mon[32697]: pgmap v144: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 KiB/s wr, 151 op/s 2024-04-26T18:29:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:03 smithi120 ceph-mon[32244]: pgmap v144: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 KiB/s wr, 151 op/s 2024-04-26T18:29:04.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.122+0000 7ff096ffd700 1 -- 172.21.15.37:0/2117546844 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564a38aa4350 con 0x564a38bcb340 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.122+0000 7ff096ffd700 10 client.25499 handle_command_reply: tid=0 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.122+0000 7ff0c8ad6b80 1 client.25499 shutdown 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 msgr2=0x564a38b58fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b58fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c340 tx=0x7ff0a8098810 comp rx=0 tx=0).stop 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 shutdown_connections 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x564a38bcb340 0x564a38bcd730 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564a38b54a00 0x564a38b58fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564a38b49230 0x564a38b58550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 --2- 172.21.15.37:0/2117546844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a38b49b50 0x564a38b58a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff096ffd700 0 client.25499 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 >> 172.21.15.37:0/2117546844 conn(0x564a38a76ed0 msgr2=0x564a38b535e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 shutdown_connections 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 1 -- 172.21.15.37:0/2117546844 wait complete. 2024-04-26T18:29:04.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.123+0000 7ff0c8ad6b80 20 client.25499 trim_cache size 0 max 16384 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 28, 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 14, 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "3add848f": { 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 14, 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:29:04.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 79.5, 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:04.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 msgr2=0x7ff0a40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 0x7ff0a40a6390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0003760 tx=0x7ff0b0005fb0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 msgr2=0x7ff0b8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c370 tx=0x7ff0a800c680 comp rx=0 tx=0).stop 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 shutdown_connections 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff0a40a3ed0 0x7ff0a40a6390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0b8107650 0x7ff0b8109d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0b8102580 0x7ff0b81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 --2- 172.21.15.37:0/3591515088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0b8102f30 0x7ff0b8109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 >> 172.21.15.37:0/3591515088 conn(0x7ff0b80792c0 msgr2=0x7ff0b8100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 shutdown_connections 2024-04-26T18:29:04.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:04.124+0000 7ff0be244700 1 -- 172.21.15.37:0/3591515088 wait complete. 2024-04-26T18:29:04.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3add848f', quiesce took 7.0 seconds. Will release after: 19.5 2024-04-26T18:29:05.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:05 smithi037 ceph-mon[29171]: pgmap v145: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 808 KiB/s wr, 144 op/s 2024-04-26T18:29:05.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:05 smithi170 ceph-mon[32697]: pgmap v145: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 808 KiB/s wr, 144 op/s 2024-04-26T18:29:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:05 smithi120 ceph-mon[32244]: pgmap v145: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 808 KiB/s wr, 144 op/s 2024-04-26T18:29:07.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:07 smithi170 ceph-mon[32697]: pgmap v146: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:07.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:07 smithi037 ceph-mon[29171]: pgmap v146: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:07 smithi120 ceph-mon[32244]: pgmap v146: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:09.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:09 smithi170 ceph-mon[32697]: pgmap v147: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:09 smithi037 ceph-mon[29171]: pgmap v147: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:09 smithi120 ceph-mon[32244]: pgmap v147: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:11.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:11 smithi037 ceph-mon[29171]: pgmap v148: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:11 smithi120 ceph-mon[32244]: pgmap v148: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:11 smithi170 ceph-mon[32697]: pgmap v148: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-04-26T18:29:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 -- 172.21.15.37:0/1302947522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 msgr2=0x7f11a810f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 --2- 172.21.15.37:0/1302947522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a810f3b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009ca0 tx=0x7f11a002a950 comp rx=0 tx=0).stop 2024-04-26T18:29:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 -- 172.21.15.37:0/1302947522 shutdown_connections 2024-04-26T18:29:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 --2- 172.21.15.37:0/1302947522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a810f3b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 --2- 172.21.15.37:0/1302947522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 0x7f11a810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 --2- 172.21.15.37:0/1302947522 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a8103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 -- 172.21.15.37:0/1302947522 >> 172.21.15.37:0/1302947522 conn(0x7f11a80789b0 msgr2=0x7f11a8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 -- 172.21.15.37:0/1302947522 shutdown_connections 2024-04-26T18:29:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.878+0000 7f11aff40640 1 -- 172.21.15.37:0/1302947522 wait complete. 2024-04-26T18:29:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 Processor -- start 2024-04-26T18:29:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 -- start start 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 0x7f11a8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112bc0 con 0x7f11a810cf30 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112a40 con 0x7f11a8103b60 2024-04-26T18:29:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11aff40640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8112d40 con 0x7f11a8104480 2024-04-26T18:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.879+0000 7f11ae4dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59352/0 (socket says 172.21.15.37:59352) 2024-04-26T18:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 -- 172.21.15.37:0/3929788383 learned_addr learned my addr 172.21.15.37:0/3929788383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ad4db640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 0x7f11a8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11adcdc640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 -- 172.21.15.37:0/3929788383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 msgr2=0x7f11a8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 0x7f11a8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 -- 172.21.15.37:0/3929788383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 msgr2=0x7f11a81017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a81017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 -- 172.21.15.37:0/3929788383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a81a7540 con 0x7f11a810cf30 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11adcdc640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a81017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11ae4dd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f11a002ae70 tx=0x7f11a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f119effd640 1 -- 172.21.15.37:0/3929788383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0034040 con 0x7f11a810cf30 2024-04-26T18:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81a77d0 con 0x7f11a810cf30 2024-04-26T18:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f119effd640 1 -- 172.21.15.37:0/3929788383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a00308f0 con 0x7f11a810cf30 2024-04-26T18:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f119effd640 1 -- 172.21.15.37:0/3929788383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00047f0 con 0x7f11a810cf30 2024-04-26T18:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.880+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a7d10 con 0x7f11a810cf30 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.883+0000 7f119effd640 1 -- 172.21.15.37:0/3929788383 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11a0037020 con 0x7f11a810cf30 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.883+0000 7f11b2c59740 1 Processor -- start 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.883+0000 7f11b2c59740 1 -- start start 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558749654460 con 0x5587496ee820 2024-04-26T18:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55874960e590 con 0x5587496f90e0 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11b2c59740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558749650420 con 0x5587496edf00 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f119effd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 0x7f11940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59366/0 (socket says 172.21.15.37:59366) 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 -- 172.21.15.37:0/2605179108 learned_addr learned my addr 172.21.15.37:0/2605179108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11ae4dd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 0x7f11940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11ad4db640 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11ad4db640 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f119effd640 1 -- 172.21.15.37:0/3929788383 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f11a00f16f0 con 0x7f11a810cf30 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 -- 172.21.15.37:0/2605179108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 msgr2=0x5587496ee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496ee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 -- 172.21.15.37:0/2605179108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 msgr2=0x5587496fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.884+0000 7f11adcdc640 1 -- 172.21.15.37:0/2605179108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55874961b0d0 con 0x5587496ee820 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.885+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f11a400bd40 tx=0x7f11a400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337e2212ede64a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.885+0000 7f11ae4dd640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 0x7f11940a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f11a00bd430 tx=0x7f11a00b9330 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.885+0000 7f1193fff640 1 -- 172.21.15.37:0/2605179108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4015070 con 0x5587496ee820 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.885+0000 7f1193fff640 1 -- 172.21.15.37:0/2605179108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a400be30 con 0x5587496ee820 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.885+0000 7f1193fff640 1 -- 172.21.15.37:0/2605179108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a40119a0 con 0x5587496ee820 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- 172.21.15.37:0/2605179108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 msgr2=0x5587496f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f11a400bd40 tx=0x7f11a400f730 comp rx=0 tx=0).stop 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- 172.21.15.37:0/2605179108 shutdown_connections 2024-04-26T18:29:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x5587496f8ba0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2605179108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496ee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- 172.21.15.37:0/2605179108 >> 172.21.15.37:0/2605179108 conn(0x5587496ea410 msgr2=0x5587496ea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- 172.21.15.37:0/2605179108 shutdown_connections 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- 172.21.15.37:0/2605179108 wait complete. 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 -- start start 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.886+0000 7f11b2c59740 1 Processor -- start 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496edf00 0x5587496f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496ee820 0x5587496f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5587496fbd10 con 0x5587496f90e0 2024-04-26T18:29:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5587495dbb60 con 0x5587496edf00 2024-04-26T18:29:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5587496fbe90 con 0x5587496ee820 2024-04-26T18:29:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11b2c59740 10 client.0 _ll_register_callbacks cb 0x5587496002b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59382/0 (socket says 172.21.15.37:59382) 2024-04-26T18:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 -- 172.21.15.37:0/1498159232 learned_addr learned my addr 172.21.15.37:0/1498159232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496edf00 0x5587496f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11adcdc640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496ee820 0x5587496f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 -- 172.21.15.37:0/1498159232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496ee820 msgr2=0x5587496f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496ee820 0x5587496f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 -- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496edf00 msgr2=0x5587496f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496edf00 0x5587496f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 -- 172.21.15.37:0/1498159232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558749366210 con 0x5587496f90e0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11ad4db640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1198012c70 tx=0x7f1198004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b40002224aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11917fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.887+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198010e70 con 0x5587496f90e0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f1198005020 con 0x5587496f90e0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a400be30 con 0x5587496f90e0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1198005220 con 0x5587496f90e0 2024-04-26T18:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558749432620 con 0x5587496f90e0 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a400be30 con 0x5587496f90e0 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f1198022480 con 0x5587496f90e0 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 20 client.15888 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558749709320 0x55874970b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 4 client.15888 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.888+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558749662070 con 0x558749709320 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.889+0000 7f11adcdc640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558749709320 0x55874970b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.889+0000 7f11adcdc640 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558749709320 0x55874970b750 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6804d05d8805c678 server_cookie=2ab20897f1dcf308 in_seq=0 out_seq=0 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.889+0000 7f11917fa640 10 client.15888 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11917fa640 1 -- 172.21.15.37:0/1498159232 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558749662070 con 0x558749709320 2024-04-26T18:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11917fa640 10 client.15888 handle_command_reply: tid=0 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 client.15888 shutdown 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 msgr2=0x5587496f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1198012c70 tx=0x7f1198004730 comp rx=0 tx=0).stop 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 shutdown_connections 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558749709320 0x55874970b750 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496f90e0 0x5587496f6b30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496ee820 0x5587496f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11917fa640 0 client.15888 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 --2- 172.21.15.37:0/1498159232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496edf00 0x5587496f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 >> 172.21.15.37:0/1498159232 conn(0x5587496ea410 msgr2=0x5587496e95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 shutdown_connections 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 1 -- 172.21.15.37:0/1498159232 wait complete. 2024-04-26T18:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.890+0000 7f11b2c59740 20 client.15888 trim_cache size 0 max 16384 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.905+0000 7f11b2c59740 1 Processor -- start 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.905+0000 7f11b2c59740 1 -- start start 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b3030 con 0x5587496ee820 2024-04-26T18:29:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b0900 con 0x5587496f90e0 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11b2c59740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81ae750 con 0x5587496edf00 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59388/0 (socket says 172.21.15.37:59388) 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 -- 172.21.15.37:0/3567381811 learned_addr learned my addr 172.21.15.37:0/3567381811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11ad4db640 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 -- 172.21.15.37:0/3567381811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 msgr2=0x5587496e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 -- 172.21.15.37:0/3567381811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 msgr2=0x5587496f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 -- 172.21.15.37:0/3567381811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587496f5080 con 0x5587496ee820 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11ad4db640 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11adcdc640 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f11a40115f0 tx=0x7f11a400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963b2a94b3e6f3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11937fe640 1 -- 172.21.15.37:0/3567381811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4015040 con 0x5587496ee820 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11937fe640 1 -- 172.21.15.37:0/3567381811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a4005570 con 0x5587496ee820 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.906+0000 7f11937fe640 1 -- 172.21.15.37:0/3567381811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4028550 con 0x5587496ee820 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- 172.21.15.37:0/3567381811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 msgr2=0x558749704bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f11a40115f0 tx=0x7f11a400b660 comp rx=0 tx=0).stop 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- 172.21.15.37:0/3567381811 shutdown_connections 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496f90e0 0x5587496f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496ee820 0x558749704bd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 --2- 172.21.15.37:0/3567381811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496edf00 0x5587496e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- 172.21.15.37:0/3567381811 >> 172.21.15.37:0/3567381811 conn(0x5587496ea410 msgr2=0x5587496ea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- 172.21.15.37:0/3567381811 shutdown_connections 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- 172.21.15.37:0/3567381811 wait complete. 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 -- start start 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.907+0000 7f11b2c59740 1 Processor -- start 2024-04-26T18:29:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 0x5587496f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496f90e0 0x55874976e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81aedc0 con 0x5587496edf00 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b0230 con 0x5587496ee820 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81af490 con 0x5587496f90e0 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 10 client.0 _ll_register_callbacks cb 0x7f119800bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59394/0 (socket says 172.21.15.37:59394) 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 -- 172.21.15.37:0/2394190056 learned_addr learned my addr 172.21.15.37:0/2394190056 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 0x5587496f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 -- 172.21.15.37:0/2394190056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496f90e0 msgr2=0x55874976e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496f90e0 0x55874976e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 -- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 msgr2=0x5587496f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 0x5587496f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 -- 172.21.15.37:0/2394190056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55874976edd0 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 0x5587496f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11ad4db640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f119800bf20 tx=0x7f119800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8735e9922c1d9070 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11917fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119802f030 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f119801b040 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.908+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a4005570 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f119800b670 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558749770050 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a4005570 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f1198022cd0 con 0x5587496edf00 2024-04-26T18:29:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 20 client.15894 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587497707e0 0x558749772c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 4 client.15894 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55874965bee0 con 0x5587497707e0 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.909+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587497707e0 0x558749772c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.910+0000 7f11917fa640 10 client.15894 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.910+0000 7f11adcdc640 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587497707e0 0x558749772c00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4d04ab81a7cdd7 server_cookie=7451e0508575c854 in_seq=0 out_seq=0 2024-04-26T18:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11917fa640 1 -- 172.21.15.37:0/2394190056 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55874965bee0 con 0x5587497707e0 2024-04-26T18:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11917fa640 10 client.15894 handle_command_reply: tid=0 2024-04-26T18:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 client.15894 shutdown 2024-04-26T18:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 msgr2=0x5587496f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f119800bf20 tx=0x7f119800bf50 comp rx=0 tx=0).stop 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 shutdown_connections 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5587497707e0 0x558749772c00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5587496ee820 0x5587496f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11917fa640 0 client.15894 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5587496f90e0 0x55874976e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 --2- 172.21.15.37:0/2394190056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5587496edf00 0x5587496f3470 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 >> 172.21.15.37:0/2394190056 conn(0x5587496ea410 msgr2=0x5587496f0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 shutdown_connections 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 1 -- 172.21.15.37:0/2394190056 wait complete. 2024-04-26T18:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.927+0000 7f11b2c59740 20 client.15894 trim_cache size 0 max 16384 2024-04-26T18:29:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:29:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:29:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:29:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 msgr2=0x7f11940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 0x7f11940a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f11a00bd430 tx=0x7f11a00b9330 comp rx=0 tx=0).stop 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 msgr2=0x7f11a8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f11a002ae70 tx=0x7f11a002f5d0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 shutdown_connections 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f11940a3e40 0x7f11940a6300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11a810cf30 0x7f11a8102230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8104480 0x7f11a8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 --2- 172.21.15.37:0/3929788383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11a8103b60 0x7f11a81017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 >> 172.21.15.37:0/3929788383 conn(0x7f11a80789b0 msgr2=0x7f11a80ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 shutdown_connections 2024-04-26T18:29:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:11.928+0000 7f11aff40640 1 -- 172.21.15.37:0/3929788383 wait complete. 2024-04-26T18:29:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:13 smithi037 ceph-mon[29171]: pgmap v149: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:13 smithi120 ceph-mon[32244]: pgmap v149: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:13 smithi170 ceph-mon[32697]: pgmap v149: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-26T18:29:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:15 smithi037 ceph-mon[29171]: pgmap v150: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 446 KiB/s wr, 157 op/s 2024-04-26T18:29:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:15 smithi120 ceph-mon[32244]: pgmap v150: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 446 KiB/s wr, 157 op/s 2024-04-26T18:29:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:15 smithi170 ceph-mon[32697]: pgmap v150: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 446 KiB/s wr, 157 op/s 2024-04-26T18:29:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:17 smithi037 ceph-mon[29171]: pgmap v151: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 500 KiB/s wr, 164 op/s 2024-04-26T18:29:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:17 smithi120 ceph-mon[32244]: pgmap v151: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 500 KiB/s wr, 164 op/s 2024-04-26T18:29:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:17 smithi170 ceph-mon[32697]: pgmap v151: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 500 KiB/s wr, 164 op/s 2024-04-26T18:29:19.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:19 smithi037 ceph-mon[29171]: pgmap v152: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:19 smithi120 ceph-mon[32244]: pgmap v152: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:19 smithi170 ceph-mon[32697]: pgmap v152: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:21 smithi037 ceph-mon[29171]: pgmap v153: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:21 smithi120 ceph-mon[32244]: pgmap v153: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:21 smithi170 ceph-mon[32697]: pgmap v153: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 54 KiB/s wr, 9 op/s 2024-04-26T18:29:23.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:23 smithi037 ceph-mon[29171]: pgmap v154: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-04-26T18:29:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:23 smithi120 ceph-mon[32244]: pgmap v154: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-04-26T18:29:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:23 smithi170 ceph-mon[32697]: pgmap v154: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-04-26T18:29:24.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:29:24.637 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 -- 172.21.15.37:0/661768219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101bc0 msgr2=0x7f5af4105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/661768219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101bc0 0x7f5af4105c00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009900 tx=0x7f5ae802a410 comp rx=0 tx=0).stop 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 -- 172.21.15.37:0/661768219 shutdown_connections 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/661768219 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af41062e0 0x7f5af410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/661768219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101bc0 0x7f5af4105c00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/661768219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af4101210 0x7f5af41015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 -- 172.21.15.37:0/661768219 >> 172.21.15.37:0/661768219 conn(0x7f5af4077dc0 msgr2=0x7f5af40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 -- 172.21.15.37:0/661768219 shutdown_connections 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 -- 172.21.15.37:0/661768219 wait complete. 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.021+0000 7f5afa0b0640 1 Processor -- start 2024-04-26T18:29:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 -- start start 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af41062e0 0x7f5af4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af419b790 0x7f5af419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41109c0 con 0x7f5af4101210 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4110840 con 0x7f5af419b790 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5afa0b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4110b40 con 0x7f5af41062e0 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43808/0 (socket says 172.21.15.37:43808) 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 -- 172.21.15.37:0/1674769000 learned_addr learned my addr 172.21.15.37:0/1674769000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af3fff640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af419b790 0x7f5af419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af41062e0 0x7f5af4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 -- 172.21.15.37:0/1674769000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af41062e0 msgr2=0x7f5af4197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af41062e0 0x7f5af4197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 -- 172.21.15.37:0/1674769000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af419b790 msgr2=0x7f5af419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af419b790 0x7f5af419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 -- 172.21.15.37:0/1674769000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af419c280 con 0x7f5af4101210 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.022+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400ffd0 tx=0x7f5ae4003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.023+0000 7f5af0ff9640 1 -- 172.21.15.37:0/1674769000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4018420 con 0x7f5af4101210 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.023+0000 7f5af0ff9640 1 -- 172.21.15.37:0/1674769000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae401d070 con 0x7f5af4101210 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.023+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af419c570 con 0x7f5af4101210 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.023+0000 7f5af0ff9640 1 -- 172.21.15.37:0/1674769000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40178e0 con 0x7f5af4101210 2024-04-26T18:29:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.023+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a3d10 con 0x7f5af4101210 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.025+0000 7f5af0ff9640 1 -- 172.21.15.37:0/1674769000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ae4018c20 con 0x7f5af4101210 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.025+0000 7f5afcdc8740 1 Processor -- start 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.025+0000 7f5afcdc8740 1 -- start start 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddcf00 0x55afb9ddd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55afb9d43460 con 0x55afb9de80e0 2024-04-26T18:29:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55afb9cfd590 con 0x55afb9ddcf00 2024-04-26T18:29:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55afb9d3f420 con 0x55afb9ddd820 2024-04-26T18:29:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af0ff9640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 0x7f5adc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddcf00 0x55afb9ddd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43814/0 (socket says 172.21.15.37:43814) 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3307809520 learned_addr learned my addr 172.21.15.37:0/3307809520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af3fff640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 0x7f5adc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af0ff9640 1 -- 172.21.15.37:0/1674769000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f5ae40d4760 con 0x7f5af4101210 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af37fe640 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3307809520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 msgr2=0x55afb9de7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3307809520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddcf00 msgr2=0x55afb9ddd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddcf00 0x55afb9ddd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3307809520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb9d0a0d0 con 0x55afb9de80e0 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.026+0000 7f5af37fe640 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.027+0000 7f5af3fff640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 0x7f5adc0a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005fd0 tx=0x7f5aec00b980 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.027+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9264bc09e00337 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.027+0000 7f5ad9ffb640 1 -- 172.21.15.37:0/3307809520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae803f070 con 0x55afb9de80e0 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.027+0000 7f5ad9ffb640 1 -- 172.21.15.37:0/3307809520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8004030 con 0x55afb9de80e0 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.027+0000 7f5ad9ffb640 1 -- 172.21.15.37:0/3307809520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8035070 con 0x55afb9de80e0 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- 172.21.15.37:0/3307809520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 msgr2=0x55afb9dea4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006010 tx=0x7f5ae8032b20 comp rx=0 tx=0).stop 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- 172.21.15.37:0/3307809520 shutdown_connections 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9dea4d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/3307809520 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddcf00 0x55afb9ddd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- 172.21.15.37:0/3307809520 >> 172.21.15.37:0/3307809520 conn(0x55afb9dd9410 msgr2=0x55afb9dd97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- 172.21.15.37:0/3307809520 shutdown_connections 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- 172.21.15.37:0/3307809520 wait complete. 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 -- start start 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.028+0000 7f5afcdc8740 1 Processor -- start 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9de5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55afb9deacb0 con 0x55afb9ddcf00 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55afb9ccab60 con 0x55afb9de80e0 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55afb9deae30 con 0x55afb9ddd820 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43822/0 (socket says 172.21.15.37:43822) 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 -- 172.21.15.37:0/1782757067 learned_addr learned my addr 172.21.15.37:0/1782757067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5afcdc8740 10 client.0 _ll_register_callbacks cb 0x55afb9cef2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af3fff640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9de5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 -- 172.21.15.37:0/1782757067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 msgr2=0x55afb9de5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 -- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 msgr2=0x55afb9de5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9de5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 -- 172.21.15.37:0/1782757067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb9a55210 con 0x55afb9ddcf00 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40d24a0 tx=0x7f5ae40a0df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f249b313c2ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.029+0000 7f5ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40a4aa0 con 0x55afb9ddcf00 2024-04-26T18:29:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae8004030 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f5ae409cd60 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ae40a9360 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afb9b21620 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8004030 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f5ae400b040 con 0x55afb9ddcf00 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 20 client.15906 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9df81e0 0x55afb9dfa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 4 client.15906 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.030+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afb9d51070 con 0x55afb9df81e0 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.031+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9df81e0 0x55afb9dfa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.031+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9df81e0 0x55afb9dfa5d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9b2d112cee6e56 server_cookie=6f583cb85695b14a in_seq=0 out_seq=0 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.031+0000 7f5ad2ffd640 10 client.15906 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/1782757067 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afb9d51070 con 0x55afb9df81e0 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5ad2ffd640 10 client.15906 handle_command_reply: tid=0 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 client.15906 shutdown 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 msgr2=0x55afb9de5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40d24a0 tx=0x7f5ae40a0df0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 shutdown_connections 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9df81e0 0x55afb9dfa5d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9de5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddd820 0x55afb9de5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5ad2ffd640 0 client.15906 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1782757067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddcf00 0x55afb9de5020 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 >> 172.21.15.37:0/1782757067 conn(0x55afb9dd9410 msgr2=0x55afb9ddf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 shutdown_connections 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1782757067 wait complete. 2024-04-26T18:29:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.032+0000 7f5afcdc8740 20 client.15906 trim_cache size 0 max 16384 2024-04-26T18:29:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 Processor -- start 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 -- start start 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9df3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9dffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41aeac0 con 0x55afb9ddd820 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41ac390 con 0x55afb9de80e0 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41aa1e0 con 0x55afb9ddcf00 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5af2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43824/0 (socket says 172.21.15.37:43824) 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.047+0000 7f5af2ffd640 1 -- 172.21.15.37:0/1062568032 learned_addr learned my addr 172.21.15.37:0/1062568032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af3fff640 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9dffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af37fe640 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9df3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 -- 172.21.15.37:0/1062568032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 msgr2=0x55afb9df3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9df3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 -- 172.21.15.37:0/1062568032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 msgr2=0x55afb9dffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9dffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 -- 172.21.15.37:0/1062568032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb9de5f80 con 0x55afb9ddd820 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80096c0 tx=0x7f5ae8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327524c707af2d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5ad97fa640 1 -- 172.21.15.37:0/1062568032 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8035070 con 0x55afb9ddd820 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5ad97fa640 1 -- 172.21.15.37:0/1062568032 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8004030 con 0x55afb9ddd820 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.048+0000 7f5ad97fa640 1 -- 172.21.15.37:0/1062568032 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae803f040 con 0x55afb9ddd820 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1062568032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 msgr2=0x55afb9ddf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80096c0 tx=0x7f5ae8009650 comp rx=0 tx=0).stop 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1062568032 shutdown_connections 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9de80e0 0x55afb9dffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9ddd820 0x55afb9ddf8c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/1062568032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9df3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1062568032 >> 172.21.15.37:0/1062568032 conn(0x55afb9dd9410 msgr2=0x55afb9de5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1062568032 shutdown_connections 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- 172.21.15.37:0/1062568032 wait complete. 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 -- start start 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.049+0000 7f5afcdc8740 1 Processor -- start 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9de2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddd820 0x55afb9de28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41aa850 con 0x55afb9de80e0 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41abcc0 con 0x55afb9ddd820 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5af41aaf20 con 0x55afb9ddcf00 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 10 client.0 _ll_register_callbacks cb 0x7f5ae409c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43826/0 (socket says 172.21.15.37:43826) 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 -- 172.21.15.37:0/2948612049 learned_addr learned my addr 172.21.15.37:0/2948612049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddd820 0x55afb9de28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af37fe640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9de2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 -- 172.21.15.37:0/2948612049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 msgr2=0x55afb9de2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9de2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 -- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddd820 msgr2=0x55afb9de28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddd820 0x55afb9de28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 -- 172.21.15.37:0/2948612049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb9e5dc40 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af37fe640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9de2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5af3fff640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0186b0 tx=0x7f5aec00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cdf5a2063db08e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01f040 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f5aec01bcf0 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.050+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae8004030 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5aec004b20 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 10 client.15912 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afb9e5ee80 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8004030 con 0x55afb9de80e0 2024-04-26T18:29:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f5aec0056f0 con 0x55afb9de80e0 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 20 client.15912 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9e5f660 0x55afb9e61a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 4 client.15912 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afb9d4aee0 con 0x55afb9e5f660 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.051+0000 7f5af37fe640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9e5f660 0x55afb9e61a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.052+0000 7f5af37fe640 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9e5f660 0x55afb9e61a80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f333d7dbd48a8ac server_cookie=4868128faf961cd9 in_seq=0 out_seq=0 2024-04-26T18:29:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.052+0000 7f5ad2ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5ad2ffd640 1 -- 172.21.15.37:0/2948612049 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afb9d4aee0 con 0x55afb9e5f660 2024-04-26T18:29:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5ad2ffd640 10 client.15912 handle_command_reply: tid=0 2024-04-26T18:29:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 client.15912 shutdown 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 msgr2=0x55afb9e5d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0186b0 tx=0x7f5aec00d040 comp rx=0 tx=0).stop 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 shutdown_connections 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55afb9e5f660 0x55afb9e61a80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55afb9ddd820 0x55afb9de28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55afb9de80e0 0x55afb9e5d700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5ad2ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 --2- 172.21.15.37:0/2948612049 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55afb9ddcf00 0x55afb9de2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.065+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 >> 172.21.15.37:0/2948612049 conn(0x55afb9dd9410 msgr2=0x55afb9dd7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 shutdown_connections 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afcdc8740 1 -- 172.21.15.37:0/2948612049 wait complete. 2024-04-26T18:29:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afcdc8740 20 client.15912 trim_cache size 0 max 16384 2024-04-26T18:29:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:29:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:29:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:29:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 msgr2=0x7f5adc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 0x7f5adc0a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005fd0 tx=0x7f5aec00b980 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 msgr2=0x7f5af4196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400ffd0 tx=0x7f5ae4003ee0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 shutdown_connections 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f5adc0a3dc0 0x7f5adc0a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5af419b790 0x7f5af419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af41062e0 0x7f5af4197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 --2- 172.21.15.37:0/1674769000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af4101210 0x7f5af4196f20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 >> 172.21.15.37:0/1674769000 conn(0x7f5af4077dc0 msgr2=0x7f5af40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 shutdown_connections 2024-04-26T18:29:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:25.066+0000 7f5afa0b0640 1 -- 172.21.15.37:0/1674769000 wait complete. 2024-04-26T18:29:25.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:25 smithi037 ceph-mon[29171]: pgmap v155: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-26T18:29:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:25 smithi120 ceph-mon[32244]: pgmap v155: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-26T18:29:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:25 smithi170 ceph-mon[32697]: pgmap v155: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 58 KiB/s wr, 9 op/s 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 -- 172.21.15.37:0/951762247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 msgr2=0x7f41481062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 --2- 172.21.15.37:0/951762247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481062f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f414000b0a0 tx=0x7f414000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 -- 172.21.15.37:0/951762247 shutdown_connections 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 --2- 172.21.15.37:0/951762247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481062f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 --2- 172.21.15.37:0/951762247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4148108380 0x7f414806bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 --2- 172.21.15.37:0/951762247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f414806c110 0x7f414806c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 -- 172.21.15.37:0/951762247 >> 172.21.15.37:0/951762247 conn(0x7f41480793c0 msgr2=0x7f41480797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 -- 172.21.15.37:0/951762247 shutdown_connections 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.344+0000 7f414ed00700 1 -- 172.21.15.37:0/951762247 wait complete. 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 Processor -- start 2024-04-26T18:29:26.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 -- start start 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f414806c110 0x7f41481a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4148108380 0x7f41481a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4148119c10 con 0x7f4148105e90 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4148119a90 con 0x7f4148108380 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ed00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4148119d90 con 0x7f414806c110 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43838/0 (socket says 172.21.15.37:43838) 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 -- 172.21.15.37:0/4264570208 learned_addr learned my addr 172.21.15.37:0/4264570208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414d29d700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4148108380 0x7f41481a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:26.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f414ca9c700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f414806c110 0x7f41481a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 -- 172.21.15.37:0/4264570208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f414806c110 msgr2=0x7f41481a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f414806c110 0x7f41481a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 -- 172.21.15.37:0/4264570208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4148108380 msgr2=0x7f41481a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4148108380 0x7f41481a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.345+0000 7f4147fff700 1 -- 172.21.15.37:0/4264570208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41481a5070 con 0x7f4148105e90 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f4147fff700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9a0 tx=0x7f413400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:26.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400cbd0 con 0x7f4148105e90 2024-04-26T18:29:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f413400cd70 con 0x7f4148105e90 2024-04-26T18:29:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41481a5300 con 0x7f4148105e90 2024-04-26T18:29:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134018680 con 0x7f4148105e90 2024-04-26T18:29:26.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.346+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41481acbe0 con 0x7f4148105e90 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.347+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414806a110 con 0x7f4148105e90 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.348+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4134010430 con 0x7f4148105e90 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.349+0000 7f4145ffb700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 0x7f41380a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.349+0000 7f414ca9c700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 0x7f41380a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.349+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f41340d0f00 con 0x7f4148105e90 2024-04-26T18:29:26.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.350+0000 7f414ca9c700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 0x7f41380a62f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f413c005fd0 tx=0x7f413c005ce0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:26.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.353+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f413409d530 con 0x7f4148105e90 2024-04-26T18:29:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.615+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41481a13c0 con 0x7f4148105e90 2024-04-26T18:29:26.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.615+0000 7f4145ffb700 1 -- 172.21.15.37:0/4264570208 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11884 (secure 0 0 0) 0x7f413409ce30 con 0x7f4148105e90 2024-04-26T18:29:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:29:26.616 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:28:50.545932+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 msgr2=0x7f41380a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 0x7f41380a62f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f413c005fd0 tx=0x7f413c005ce0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 msgr2=0x7f41481a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9a0 tx=0x7f413400ecb0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 shutdown_connections 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f41380a3e30 0x7f41380a62f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4148108380 0x7f41481a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f414806c110 0x7f41481a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 --2- 172.21.15.37:0/4264570208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4148105e90 0x7f41481a05a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 >> 172.21.15.37:0/4264570208 conn(0x7f41480793c0 msgr2=0x7f4148104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 shutdown_connections 2024-04-26T18:29:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:26.618+0000 7f414ed00700 1 -- 172.21.15.37:0/4264570208 wait complete. 2024-04-26T18:29:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 28 2024-04-26T18:29:27.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:27 smithi037 ceph-mon[29171]: pgmap v156: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 120 KiB/s wr, 16 op/s 2024-04-26T18:29:27.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:27 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4264570208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:27.351 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --set-id 3add848f --release --await' 2024-04-26T18:29:27.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db --set-id 3add848f --release --await 2024-04-26T18:29:27.506 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:27 smithi120 ceph-mon[32244]: pgmap v156: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 120 KiB/s wr, 16 op/s 2024-04-26T18:29:27.506 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:27 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4264570208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:27 smithi170 ceph-mon[32697]: pgmap v156: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 120 KiB/s wr, 16 op/s 2024-04-26T18:29:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:27 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4264570208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:27.604 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:29.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:29 smithi037 ceph-mon[29171]: pgmap v157: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 -- 172.21.15.37:0/815491823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103e30 msgr2=0x7fa4b8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/815491823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103e30 0x7fa4b8110290 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000b0a0 tx=0x7fa4b000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 -- 172.21.15.37:0/815491823 shutdown_connections 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/815491823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103e30 0x7fa4b8110290 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/815491823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 0x7fa4b81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.244+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/815491823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103470 0x7fa4b81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- 172.21.15.37:0/815491823 >> 172.21.15.37:0/815491823 conn(0x7fa4b807a580 msgr2=0x7fa4b807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- 172.21.15.37:0/815491823 shutdown_connections 2024-04-26T18:29:29.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- 172.21.15.37:0/815491823 wait complete. 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 Processor -- start 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- start start 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103e30 0x7fa4b819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 0x7fa4b8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8113640 con 0x7fa4b8103470 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81134c0 con 0x7fa4b81094c0 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.245+0000 7fa4bf18a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81137c0 con 0x7fa4b8103e30 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43856/0 (socket says 172.21.15.37:43856) 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1296787140 learned_addr learned my addr 172.21.15.37:0/1296787140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103e30 0x7fa4b819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bd727700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 0x7fa4b8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1296787140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103e30 msgr2=0x7fa4b819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103e30 0x7fa4b819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1296787140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 msgr2=0x7fa4b8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 0x7fa4b8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1296787140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b819a0a0 con 0x7fa4b8103470 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00d950 tx=0x7fa4ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4b5ffb700 1 -- 172.21.15.37:0/1296787140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac016070 con 0x7fa4b8103470 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b819a390 con 0x7fa4b8103470 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4b5ffb700 1 -- 172.21.15.37:0/1296787140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac00fd40 con 0x7fa4b8103470 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.246+0000 7fa4b5ffb700 1 -- 172.21.15.37:0/1296787140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00e3b0 con 0x7fa4b8103470 2024-04-26T18:29:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.247+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a6650 con 0x7fa4b8103470 2024-04-26T18:29:29.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.249+0000 7fa4b5ffb700 1 -- 172.21.15.37:0/1296787140 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4ac019070 con 0x7fa4b8103470 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.249+0000 7fa4b5ffb700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 0x7fa4a80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.249+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 0x7fa4a80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.249+0000 7fa4b5ffb700 1 -- 172.21.15.37:0/1296787140 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa4ac0d0650 con 0x7fa4b8103470 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 0x7fa4a80a6270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4b807abf0 tx=0x7fa4a4009190 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 Processor -- start 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 -- start start 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.250+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640c0215840 con 0x5640c02c21a0 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5640c01b46e0 con 0x5640c02c2ac0 2024-04-26T18:29:29.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5640c0211800 con 0x5640c02cd970 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43864/0 (socket says 172.21.15.37:43864) 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1095620367 learned_addr learned my addr 172.21.15.37:0/1095620367 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bd727700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1095620367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 msgr2=0x5640c02cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1095620367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 msgr2=0x5640c02cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 -- 172.21.15.37:0/1095620367 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640c01629d0 con 0x5640c02c21a0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bd727700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a4d20 tx=0x7fa4ac09cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c7a5913f72ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4a2ffd700 1 -- 172.21.15.37:0/1095620367 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac020070 con 0x5640c02c21a0 2024-04-26T18:29:29.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4a2ffd700 1 -- 172.21.15.37:0/1095620367 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac0a0990 con 0x5640c02c21a0 2024-04-26T18:29:29.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.251+0000 7fa4a2ffd700 1 -- 172.21.15.37:0/1095620367 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac09ca40 con 0x5640c02c21a0 2024-04-26T18:29:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/1095620367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 msgr2=0x5640c02c2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a4d20 tx=0x7fa4ac09cdb0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/1095620367 shutdown_connections 2024-04-26T18:29:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02c2580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/1095620367 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/1095620367 >> 172.21.15.37:0/1095620367 conn(0x5640c01efe40 msgr2=0x5640c02be690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/1095620367 shutdown_connections 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/1095620367 wait complete. 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.252+0000 7fa4c9a1cb80 1 -- start start 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 Processor -- start 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c02ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640c02d0480 con 0x5640c02c2ac0 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5640c017ba50 con 0x5640c02cd970 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5640c02d0600 con 0x5640c02c21a0 2024-04-26T18:29:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4c9a1cb80 10 client.0 _ll_register_callbacks cb 0x5640c01a9680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43872/0 (socket says 172.21.15.37:43872) 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 -- 172.21.15.37:0/3934192280 learned_addr learned my addr 172.21.15.37:0/3934192280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4bd727700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c02ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 -- 172.21.15.37:0/3934192280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 msgr2=0x5640c02cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 -- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 msgr2=0x5640c02ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c02ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.253+0000 7fa4b7fff700 1 -- 172.21.15.37:0/3934192280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640c02cb230 con 0x5640c02c2ac0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4015500 tx=0x7fa4a4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d738644811a14210 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa497fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4025070 con 0x5640c02c2ac0 2024-04-26T18:29:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa4a4015bd0 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac0a0990 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4a401b040 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa4c9a1cb80 10 client.15930 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640c02df490 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.254+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac0a0990 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fa4a40065b0 con 0x5640c02c2ac0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 10 client.15930 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 20 client.15930 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c02e1a70 0x5640c02e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 4 client.15930 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640c0223220 con 0x5640c02e1a70 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c02e1a70 0x5640c02e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa497fff700 10 client.15930 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.255+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c02e1a70 0x5640c02e3e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638d20a5fe81cec server_cookie=9969f3cf90661163 in_seq=0 out_seq=0 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa497fff700 1 -- 172.21.15.37:0/3934192280 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640c0223220 con 0x5640c02e1a70 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa497fff700 10 client.15930 handle_command_reply: tid=0 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 client.15930 shutdown 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 msgr2=0x5640c02ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4015500 tx=0x7fa4a4004860 comp rx=0 tx=0).stop 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 shutdown_connections 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c02e1a70 0x5640c02e3e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c02ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa497fff700 0 client.15930 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/3934192280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02ca4b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 >> 172.21.15.37:0/3934192280 conn(0x5640c01efe40 msgr2=0x5640c02be2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 shutdown_connections 2024-04-26T18:29:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.256+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/3934192280 wait complete. 2024-04-26T18:29:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.257+0000 7fa4c9a1cb80 20 client.15930 trim_cache size 0 max 16384 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.276+0000 7fa4c9a1cb80 1 Processor -- start 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.276+0000 7fa4c9a1cb80 1 -- start start 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02d9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ae490 con 0x5640c02c21a0 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81abd20 con 0x5640c02c2ac0 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81a9b90 con 0x5640c02cd970 2024-04-26T18:29:29.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43880/0 (socket says 172.21.15.37:43880) 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 -- 172.21.15.37:0/37312794 learned_addr learned my addr 172.21.15.37:0/37312794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02d9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bd727700 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 -- 172.21.15.37:0/37312794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 msgr2=0x5640c02d98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02d98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 -- 172.21.15.37:0/37312794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 msgr2=0x5640c02d9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02d9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 -- 172.21.15.37:0/37312794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640c02e44f0 con 0x5640c02c21a0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0ce390 tx=0x7fa4ac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6094298bbe7a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4a27fc700 1 -- 172.21.15.37:0/37312794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b9070 con 0x5640c02c21a0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4a27fc700 1 -- 172.21.15.37:0/37312794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac098c50 con 0x5640c02c21a0 2024-04-26T18:29:29.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.277+0000 7fa4a27fc700 1 -- 172.21.15.37:0/37312794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac020040 con 0x5640c02c21a0 2024-04-26T18:29:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/37312794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 msgr2=0x5640c02cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0ce390 tx=0x7fa4ac010000 comp rx=0 tx=0).stop 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/37312794 shutdown_connections 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02cd970 0x5640c02d98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c21a0 0x5640c02cc6c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/37312794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02c2ac0 0x5640c02d9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/37312794 >> 172.21.15.37:0/37312794 conn(0x5640c01efe40 msgr2=0x5640c02c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/37312794 shutdown_connections 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/37312794 wait complete. 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.278+0000 7fa4c9a1cb80 1 -- start start 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 Processor -- start 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c0346f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81aa1e0 con 0x5640c02c2ac0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81ab650 con 0x5640c02cd970 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b81aa8b0 con 0x5640c02c21a0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4c9a1cb80 10 client.0 _ll_register_callbacks cb 0x5640c02becf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43882/0 (socket says 172.21.15.37:43882) 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 -- 172.21.15.37:0/4196913824 learned_addr learned my addr 172.21.15.37:0/4196913824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4bd727700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c0346f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 -- 172.21.15.37:0/4196913824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 msgr2=0x5640c02d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02d1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 -- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 msgr2=0x5640c0346f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.279+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c0346f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa4b7fff700 1 -- 172.21.15.37:0/4196913824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640c0347550 con 0x5640c02c2ac0 2024-04-26T18:29:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa4b7fff700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40069d0 tx=0x7fa4a4013870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f330da5a4bdf3940 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4025040 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa4a4013df0 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac098c50 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4a4015810 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa4c9a1cb80 10 client.15936 fetch_fsmap learned FSMap version 28 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640c0348870 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac098c50 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.280+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fa4a402a440 con 0x5640c02c2ac0 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 10 client.15936 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 20 client.15936 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c0348fa0 0x5640c034b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 4 client.15936 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3add848f", "release": true, "await": true}] 2024-04-26T18:29:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3add848f", "release": true, "await": true}) -- 0x5640c021d2c0 con 0x5640c0348fa0 2024-04-26T18:29:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c0348fa0 0x5640c034b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa4bcf26700 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c0348fa0 0x5640c034b3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6416ba080fc58d39 server_cookie=1a620d5e4d2f9588 in_seq=0 out_seq=0 2024-04-26T18:29:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.281+0000 7fa497fff700 10 client.15936 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa497fff700 1 -- 172.21.15.37:0/4196913824 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5640c021d2c0 con 0x5640c0348fa0 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa497fff700 10 client.15936 handle_command_reply: tid=0 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 client.15936 shutdown 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 msgr2=0x5640c02d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40069d0 tx=0x7fa4a4013870 comp rx=0 tx=0).stop 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 shutdown_connections 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5640c0348fa0 0x5640c034b3c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5640c02cd970 0x5640c0346f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5640c02c21a0 0x5640c02d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 --2- 172.21.15.37:0/4196913824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640c02c2ac0 0x5640c02d1fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa497fff700 0 client.15936 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 >> 172.21.15.37:0/4196913824 conn(0x5640c01efe40 msgr2=0x5640c02c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 shutdown_connections 2024-04-26T18:29:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 1 -- 172.21.15.37:0/4196913824 wait complete. 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.288+0000 7fa4c9a1cb80 20 client.15936 trim_cache size 0 max 16384 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 28, 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "3add848f": { 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 16, 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T18:29:29.290 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 79.5, 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:29.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 msgr2=0x7fa4a80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 0x7fa4a80a6270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa4b807abf0 tx=0x7fa4a4009190 comp rx=0 tx=0).stop 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 msgr2=0x7fa4b819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00d950 tx=0x7fa4ac00dc60 comp rx=0 tx=0).stop 2024-04-26T18:29:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 shutdown_connections 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa4a80a3db0 0x7fa4a80a6270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b81094c0 0x7fa4b8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8103470 0x7fa4b819f780 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 --2- 172.21.15.37:0/1296787140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b8103e30 0x7fa4b819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 >> 172.21.15.37:0/1296787140 conn(0x7fa4b807a580 msgr2=0x7fa4b8102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 shutdown_connections 2024-04-26T18:29:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:29:29.289+0000 7fa4bf18a700 1 -- 172.21.15.37:0/1296787140 wait complete. 2024-04-26T18:29:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:29 smithi120 ceph-mon[32244]: pgmap v157: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:29 smithi170 ceph-mon[32697]: pgmap v157: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:30.063 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3add848f', seconds elapsed: 32.2 2024-04-26T18:29:30.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:30 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:30.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:30 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:30.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:30 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:31 smithi037 ceph-mon[29171]: pgmap v158: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:31 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:31 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:31 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:29:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:31 smithi120 ceph-mon[32244]: pgmap v158: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:31.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:31 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:31.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:31 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:31.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:31 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:29:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:31 smithi170 ceph-mon[32697]: pgmap v158: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 66 KiB/s wr, 9 op/s 2024-04-26T18:29:31.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:31 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:31.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:31 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:31.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:31 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:29:33.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:33 smithi037 ceph-mon[29171]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 363 KiB/s wr, 127 op/s 2024-04-26T18:29:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:33 smithi120 ceph-mon[32244]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 363 KiB/s wr, 127 op/s 2024-04-26T18:29:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:33 smithi170 ceph-mon[32697]: pgmap v159: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 363 KiB/s wr, 127 op/s 2024-04-26T18:29:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:35 smithi037 ceph-mon[29171]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 359 KiB/s wr, 124 op/s 2024-04-26T18:29:35.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:35 smithi120 ceph-mon[32244]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 359 KiB/s wr, 124 op/s 2024-04-26T18:29:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:35 smithi170 ceph-mon[32697]: pgmap v160: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 359 KiB/s wr, 124 op/s 2024-04-26T18:29:37.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:37 smithi037 ceph-mon[29171]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-04-26T18:29:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:37 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:29:37.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:37 smithi120 ceph-mon[32244]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-04-26T18:29:37.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:37 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:29:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:37 smithi170 ceph-mon[32697]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 604 op/s 2024-04-26T18:29:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:37 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:29:37.654 INFO:tasks.workunit.client.0.smithi037.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-26T18:29:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 -- 172.21.15.37:0/394602707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 msgr2=0x7f7810102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 --2- 172.21.15.37:0/394602707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f7810102db0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009930 tx=0x7f77f802a420 comp rx=0 tx=0).stop 2024-04-26T18:29:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 -- 172.21.15.37:0/394602707 shutdown_connections 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 --2- 172.21.15.37:0/394602707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78101032f0 0x7f781010f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 --2- 172.21.15.37:0/394602707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f7810102db0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 --2- 172.21.15.37:0/394602707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7810108940 0x7f7810108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 -- 172.21.15.37:0/394602707 >> 172.21.15.37:0/394602707 conn(0x7f78100fe400 msgr2=0x7f7810100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 -- 172.21.15.37:0/394602707 shutdown_connections 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 -- 172.21.15.37:0/394602707 wait complete. 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.177+0000 7f78151bd640 1 Processor -- start 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 -- start start 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f78101990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78101032f0 0x7f78101995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7810112c50 con 0x7f7810102930 2024-04-26T18:29:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7810112ad0 con 0x7f78101032f0 2024-04-26T18:29:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f78151bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7810112dd0 con 0x7f7810108940 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f780f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f780f59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:43620/0 (socket says 172.21.15.37:43620) 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f780f59e640 1 -- 172.21.15.37:0/4259088822 learned_addr learned my addr 172.21.15.37:0/4259088822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.178+0000 7f780ed9d640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f78101990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 -- 172.21.15.37:0/4259088822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78101032f0 msgr2=0x7f78101995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78101032f0 0x7f78101995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 -- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 msgr2=0x7f78101990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f78101990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 -- 172.21.15.37:0/4259088822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781019e0a0 con 0x7f7810108940 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f780f59e640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f780800eb10 tx=0x7f780800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f7807fff640 1 -- 172.21.15.37:0/4259088822 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808004b20 con 0x7f7810108940 2024-04-26T18:29:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.179+0000 7f7807fff640 1 -- 172.21.15.37:0/4259088822 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7808004cc0 con 0x7f7810108940 2024-04-26T18:29:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.180+0000 7f7807fff640 1 -- 172.21.15.37:0/4259088822 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808010670 con 0x7f7810108940 2024-04-26T18:29:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.180+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781019e390 con 0x7f7810108940 2024-04-26T18:29:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.180+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101a5c20 con 0x7f7810108940 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 Processor -- start 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7807fff640 1 -- 172.21.15.37:0/4259088822 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7808005020 con 0x7f7810108940 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 -- start start 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 0x5561f354e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f355b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561f34b4460 con 0x5561f35590e0 2024-04-26T18:29:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561f346e590 con 0x5561f354df00 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f7817fb0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561f34b0420 con 0x5561f354e820 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f780e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 0x5561f354e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:43636/0 (socket says 172.21.15.37:43636) 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.183+0000 7f780ed9d640 1 -- 172.21.15.37:0/886857979 learned_addr learned my addr 172.21.15.37:0/886857979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780e59c640 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f355b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 -- 172.21.15.37:0/886857979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 msgr2=0x5561f354e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 0x5561f354e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 -- 172.21.15.37:0/886857979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 msgr2=0x5561f355b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f355b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f7807fff640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 0x7f77fc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 -- 172.21.15.37:0/886857979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f347b0d0 con 0x5561f354e820 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780e59c640 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 0x5561f354e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780f59e640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 0x7f77fc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.184+0000 7f780ed9d640 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7800009c70 tx=0x7f7800011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52f01a02d6b8698a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.185+0000 7f7807fff640 1 -- 172.21.15.37:0/4259088822 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f78080d0d10 con 0x7f7810108940 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.185+0000 7f78057fa640 1 -- 172.21.15.37:0/886857979 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780001e070 con 0x5561f354e820 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.185+0000 7f78057fa640 1 -- 172.21.15.37:0/886857979 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7800009e20 con 0x5561f354e820 2024-04-26T18:29:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.185+0000 7f78057fa640 1 -- 172.21.15.37:0/886857979 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800019690 con 0x5561f354e820 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.185+0000 7f780f59e640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 0x7f77fc0a63e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78080ce500 tx=0x7f7808092130 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- 172.21.15.37:0/886857979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 msgr2=0x5561f3558ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7800009c70 tx=0x7f7800011810 comp rx=0 tx=0).stop 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- 172.21.15.37:0/886857979 shutdown_connections 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f355b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3558ba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 --2- 172.21.15.37:0/886857979 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354df00 0x5561f354e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- 172.21.15.37:0/886857979 >> 172.21.15.37:0/886857979 conn(0x5561f354a410 msgr2=0x5561f354a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- 172.21.15.37:0/886857979 shutdown_connections 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- 172.21.15.37:0/886857979 wait complete. 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 -- start start 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.186+0000 7f7817fb0740 1 Processor -- start 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354e820 0x5561f3556620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f3556b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5561f355bd10 con 0x5561f354e820 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561f343bb60 con 0x5561f35590e0 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561f355be90 con 0x5561f354df00 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f7817fb0740 10 client.0 _ll_register_callbacks cb 0x5561f34602b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f3556b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:43638/0 (socket says 172.21.15.37:43638) 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 -- 172.21.15.37:0/2974386194 learned_addr learned my addr 172.21.15.37:0/2974386194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780ed9d640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354e820 0x5561f3556620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 -- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 msgr2=0x5561f3556b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f3556b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 -- 172.21.15.37:0/2974386194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354e820 msgr2=0x5561f3556620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354e820 0x5561f3556620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.187+0000 7f780e59c640 1 -- 172.21.15.37:0/2974386194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f31c6210 con 0x5561f354df00 2024-04-26T18:29:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f780e59c640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c25b400ae4807 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8035070 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7800009e20 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f77f8005030 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77f8051050 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 10 client.25504 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f3292620 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7800009e20 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f77f803a040 con 0x5561f354df00 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 10 client.25504 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 10 client.25504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 20 client.25504 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f356d690 0x5561f356fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 4 client.25504 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.188+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f34c2070 con 0x5561f356d690 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.189+0000 7f780ed9d640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f356d690 0x5561f356fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.189+0000 7f780ed9d640 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f356d690 0x5561f356fa80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67eab776a5bc5e97 server_cookie=c0f18f57e2308c31 in_seq=0 out_seq=0 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.189+0000 7f77f67fc640 10 client.25504 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f77f67fc640 1 -- 172.21.15.37:0/2974386194 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561f34c2070 con 0x5561f356d690 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f77f67fc640 10 client.25504 handle_command_reply: tid=0 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 client.25504 shutdown 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 msgr2=0x5561f35560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009230 tx=0x7f77f8005fd0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 shutdown_connections 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f356d690 0x5561f356fa80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f3556b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354e820 0x5561f3556620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f77f67fc640 0 client.25504 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 --2- 172.21.15.37:0/2974386194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f35560e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 >> 172.21.15.37:0/2974386194 conn(0x5561f354a410 msgr2=0x5561f3550140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 shutdown_connections 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 1 -- 172.21.15.37:0/2974386194 wait complete. 2024-04-26T18:29:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.190+0000 7f7817fb0740 20 client.25504 trim_cache size 0 max 16384 2024-04-26T18:29:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.207+0000 7f7817fb0740 1 Processor -- start 2024-04-26T18:29:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.207+0000 7f7817fb0740 1 -- start start 2024-04-26T18:29:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f780e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f780e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49822/0 (socket says 172.21.15.37:49822) 2024-04-26T18:29:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f780e59c640 1 -- 172.21.15.37:0/3017184523 learned_addr learned my addr 172.21.15.37:0/3017184523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3556930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.208+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f354ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78101aecf0 con 0x5561f354df00 2024-04-26T18:29:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ac5c0 con 0x5561f35590e0 2024-04-26T18:29:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780ed9d640 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3556930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78101aa430 con 0x5561f354e820 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f354ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 -- 172.21.15.37:0/3017184523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 msgr2=0x5561f3556930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3556930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 -- 172.21.15.37:0/3017184523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 msgr2=0x5561f354ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f354ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.209+0000 7f780e59c640 1 -- 172.21.15.37:0/3017184523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f346e7e0 con 0x5561f354df00 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.210+0000 7f780e59c640 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f77f8003690 tx=0x7f77f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68418c4f2dc0af05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.210+0000 7f7804ff9640 1 -- 172.21.15.37:0/3017184523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f805a070 con 0x5561f354df00 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.210+0000 7f7804ff9640 1 -- 172.21.15.37:0/3017184523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f80036c0 con 0x5561f354df00 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.210+0000 7f7804ff9640 1 -- 172.21.15.37:0/3017184523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8035070 con 0x5561f354df00 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 msgr2=0x5561f3549d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f77f8003690 tx=0x7f77f8000bc0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 shutdown_connections 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f35590e0 0x5561f354ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354e820 0x5561f3556930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 --2- 172.21.15.37:0/3017184523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f354df00 0x5561f3549d30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 >> 172.21.15.37:0/3017184523 conn(0x5561f354a410 msgr2=0x5561f354f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 shutdown_connections 2024-04-26T18:29:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.211+0000 7f7817fb0740 1 -- 172.21.15.37:0/3017184523 wait complete. 2024-04-26T18:29:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 -- start start 2024-04-26T18:29:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 Processor -- start 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f3553020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78101aaa80 con 0x5561f35590e0 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78101abef0 con 0x5561f354e820 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ab150 con 0x5561f354df00 2024-04-26T18:29:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f7817fb0740 10 client.0 _ll_register_callbacks cb 0x7f77f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.212+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f3553020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:41830/0 (socket says 172.21.15.37:41830) 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49834/0 (socket says 172.21.15.37:49834) 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 -- 172.21.15.37:0/240672793 learned_addr learned my addr 172.21.15.37:0/240672793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 -- 172.21.15.37:0/240672793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 msgr2=0x5561f3553020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f3553020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 -- 172.21.15.37:0/240672793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 msgr2=0x5561f3553aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 -- 172.21.15.37:0/240672793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f35d2db0 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780e59c640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f780ed9d640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7800009bc0 tx=0x7f7800005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658cfa4afed66b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780001e040 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f78000053c0 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.213+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f80036c0 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7800003e00 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 10 client.25568 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f35d40d0 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f80036c0 con 0x5561f354e820 2024-04-26T18:29:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f780002b2f0 con 0x5561f354e820 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 10 client.25568 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 20 client.25568 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f35d4610 0x5561f35d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 4 client.25568 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561f34bbee0 con 0x5561f35d4610 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.214+0000 7f780e59c640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f35d4610 0x5561f35d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.215+0000 7f77f67fc640 10 client.25568 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.215+0000 7f780e59c640 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f35d4610 0x5561f35d6a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb194d0367bcd0cd server_cookie=55fe34d2f0849ccc in_seq=0 out_seq=0 2024-04-26T18:29:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.226+0000 7f77f67fc640 1 -- 172.21.15.37:0/240672793 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561f34bbee0 con 0x5561f35d4610 2024-04-26T18:29:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.226+0000 7f77f67fc640 10 client.25568 handle_command_reply: tid=0 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.226+0000 7f7817fb0740 1 client.25568 shutdown 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 msgr2=0x5561f3553560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7800009bc0 tx=0x7f7800005da0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 shutdown_connections 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5561f35d4610 0x5561f35d6a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561f354e820 0x5561f3553560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5561f35590e0 0x5561f3553aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 --2- 172.21.15.37:0/240672793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561f354df00 0x5561f3553020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f77f67fc640 0 client.25568 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 >> 172.21.15.37:0/240672793 conn(0x5561f354a410 msgr2=0x5561f35567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 shutdown_connections 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 1 -- 172.21.15.37:0/240672793 wait complete. 2024-04-26T18:29:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f7817fb0740 20 client.25568 trim_cache size 0 max 16384 2024-04-26T18:29:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:29:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:29:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:29:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 msgr2=0x7f77fc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 0x7f77fc0a63e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78080ce500 tx=0x7f7808092130 comp rx=0 tx=0).stop 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 msgr2=0x7f781019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f780800eb10 tx=0x7f780800c2d0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 shutdown_connections 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f77fc0a3f20 0x7f77fc0a63e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78101032f0 0x7f78101995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7810108940 0x7f781019d970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 --2- 172.21.15.37:0/4259088822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7810102930 0x7f78101990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.227+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 >> 172.21.15.37:0/4259088822 conn(0x7f78100fe400 msgr2=0x7f78100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.228+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 shutdown_connections 2024-04-26T18:29:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:38.228+0000 7f78151bd640 1 -- 172.21.15.37:0/4259088822 wait complete. 2024-04-26T18:29:39.411 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:39 smithi037 ceph-mon[29171]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:39 smithi120 ceph-mon[32244]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:39 smithi170 ceph-mon[32697]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:41 smithi037 ceph-mon[29171]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:41 smithi120 ceph-mon[32244]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:41 smithi170 ceph-mon[32697]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 597 op/s 2024-04-26T18:29:43.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:43 smithi037 ceph-mon[29171]: pgmap v164: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2024-04-26T18:29:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:43 smithi120 ceph-mon[32244]: pgmap v164: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2024-04-26T18:29:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:43 smithi170 ceph-mon[32697]: pgmap v164: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2024-04-26T18:29:43.684 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x63e5d2a 2024-04-26T18:29:45.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:45 smithi037 ceph-mon[29171]: pgmap v165: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 960 op/s 2024-04-26T18:29:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:45 smithi120 ceph-mon[32244]: pgmap v165: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 960 op/s 2024-04-26T18:29:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:45 smithi170 ceph-mon[32697]: pgmap v165: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 960 op/s 2024-04-26T18:29:46.062 INFO:tasks.workunit.client.0.smithi037.stdout:truncating to largest ever: 0x63e9b52 2024-04-26T18:29:47.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:47 smithi037 ceph-mon[29171]: pgmap v166: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.41k op/s 2024-04-26T18:29:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:47 smithi120 ceph-mon[32244]: pgmap v166: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.41k op/s 2024-04-26T18:29:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:47 smithi170 ceph-mon[32697]: pgmap v166: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 3.8 MiB/s wr, 1.41k op/s 2024-04-26T18:29:47.905 INFO:tasks.workunit.client.0.smithi037.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-26T18:29:49.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:49 smithi037 ceph-mon[29171]: pgmap v167: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:49 smithi120 ceph-mon[32244]: pgmap v167: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:49 smithi170 ceph-mon[32697]: pgmap v167: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:49.703 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x63ed6b9 2024-04-26T18:29:51.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- 172.21.15.37:0/2720951878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 msgr2=0x7f0038107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 --2- 172.21.15.37:0/2720951878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f0038107c40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0028009930 tx=0x7f002802a420 comp rx=0 tx=0).stop 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- 172.21.15.37:0/2720951878 shutdown_connections 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 --2- 172.21.15.37:0/2720951878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038102210 0x7f003810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 --2- 172.21.15.37:0/2720951878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038101850 0x7f0038101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 --2- 172.21.15.37:0/2720951878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f0038107c40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- 172.21.15.37:0/2720951878 >> 172.21.15.37:0/2720951878 conn(0x7f0038077bb0 msgr2=0x7f0038077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- 172.21.15.37:0/2720951878 shutdown_connections 2024-04-26T18:29:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- 172.21.15.37:0/2720951878 wait complete. 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 Processor -- start 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.339+0000 7f003e714640 1 -- start start 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f00381a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0038111b20 con 0x7f0038107860 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00381119a0 con 0x7f0038101850 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003e714640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0038111ca0 con 0x7f0038102210 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f0037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f00381a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:48640/0 (socket says 172.21.15.37:48640) 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f0037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:33036/0 (socket says 172.21.15.37:33036) 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 -- 172.21.15.37:0/4026309008 learned_addr learned my addr 172.21.15.37:0/4026309008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 -- 172.21.15.37:0/4026309008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 msgr2=0x7f0038199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 -- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 msgr2=0x7f00381a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f00381a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f00377fe640 1 -- 172.21.15.37:0/4026309008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381a54c0 con 0x7f0038102210 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f0037fff640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.340+0000 7f003ccb1640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f00381a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f00377fe640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f002000e9f0 tx=0x7f002000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f00357fa640 1 -- 172.21.15.37:0/4026309008 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002000cd80 con 0x7f0038102210 2024-04-26T18:29:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381a5690 con 0x7f0038102210 2024-04-26T18:29:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f00357fa640 1 -- 172.21.15.37:0/4026309008 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00200054d0 con 0x7f0038102210 2024-04-26T18:29:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f00357fa640 1 -- 172.21.15.37:0/4026309008 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020010690 con 0x7f0038102210 2024-04-26T18:29:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.341+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381a5b00 con 0x7f0038102210 2024-04-26T18:29:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.343+0000 7f00357fa640 1 -- 172.21.15.37:0/4026309008 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0020004e90 con 0x7f0038102210 2024-04-26T18:29:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.344+0000 7f004142d740 1 Processor -- start 2024-04-26T18:29:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.344+0000 7f004142d740 1 -- start start 2024-04-26T18:29:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e5151c460 con 0x556e515c03d0 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e514d6590 con 0x556e515c3fa0 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f004142d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e51518420 con 0x556e515bfa10 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:33042/0 (socket says 172.21.15.37:33042) 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f003ccb1640 1 -- 172.21.15.37:0/17452757 learned_addr learned my addr 172.21.15.37:0/17452757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f0037fff640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f00377fe640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f00357fa640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 0x7f00240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f0037fff640 1 -- 172.21.15.37:0/17452757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 msgr2=0x556e515c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f0037fff640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.345+0000 7f0037fff640 1 -- 172.21.15.37:0/17452757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 msgr2=0x556e515cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.346+0000 7f00357fa640 1 -- 172.21.15.37:0/4026309008 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f00200d0c10 con 0x7f0038102210 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.346+0000 7f003ccb1640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 0x7f00240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.346+0000 7f0037fff640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.346+0000 7f0037fff640 1 -- 172.21.15.37:0/17452757 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e514e30d0 con 0x556e515bfa10 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.346+0000 7f00377fe640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.347+0000 7f0037fff640 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515bfe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0028031440 tx=0x7f0028032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654152ba3238c7df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.347+0000 7f003ccb1640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 0x7f00240a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0030009fd0 tx=0x7f0030004ab0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.347+0000 7f002e7fc640 1 -- 172.21.15.37:0/17452757 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028035040 con 0x556e515bfa10 2024-04-26T18:29:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.347+0000 7f002e7fc640 1 -- 172.21.15.37:0/17452757 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00280317a0 con 0x556e515bfa10 2024-04-26T18:29:51.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.347+0000 7f002e7fc640 1 -- 172.21.15.37:0/17452757 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028003df0 con 0x556e515bfa10 2024-04-26T18:29:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- 172.21.15.37:0/17452757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 msgr2=0x556e515bfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515bfe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0028031440 tx=0x7f0028032ed0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- 172.21.15.37:0/17452757 shutdown_connections 2024-04-26T18:29:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515bfe90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 --2- 172.21.15.37:0/17452757 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- 172.21.15.37:0/17452757 >> 172.21.15.37:0/17452757 conn(0x556e515b1ec0 msgr2=0x556e515b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- 172.21.15.37:0/17452757 shutdown_connections 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- 172.21.15.37:0/17452757 wait complete. 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.348+0000 7f004142d740 1 -- start start 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 Processor -- start 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e515cc430 con 0x556e515c03d0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e514a3b60 con 0x556e515c3fa0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556e515cc5b0 con 0x556e515bfa10 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f004142d740 10 client.0 _ll_register_callbacks cb 0x556e514c82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:33048/0 (socket says 172.21.15.37:33048) 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f00377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f003ccb1640 1 -- 172.21.15.37:0/3216290806 learned_addr learned my addr 172.21.15.37:0/3216290806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 -- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 msgr2=0x556e515c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 -- 172.21.15.37:0/3216290806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 msgr2=0x556e515c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f0037fff640 1 -- 172.21.15.37:0/3216290806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5122e210 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.349+0000 7f00377fe640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f0037fff640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515c9340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0028005c40 tx=0x7f0028003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d74eb21b84a03e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f00177fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002804e070 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0028003800 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00280317a0 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f002805a050 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f004142d740 10 client.25522 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e512fa620 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.350+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00280317a0 con 0x556e515bfa10 2024-04-26T18:29:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f002803c7f0 con 0x556e515bfa10 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 10 client.25522 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 10 client.25522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 20 client.25522 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e515d9810 0x556e515dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 4 client.25522 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e5152a070 con 0x556e515d9810 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f00377fe640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e515d9810 0x556e515dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f00377fe640 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e515d9810 0x556e515dbc00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ff246f17681ae8 server_cookie=5536f1b93f952a21 in_seq=0 out_seq=0 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.351+0000 7f00177fe640 10 client.25522 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f00177fe640 1 -- 172.21.15.37:0/3216290806 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e5152a070 con 0x556e515d9810 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f00177fe640 10 client.25522 handle_command_reply: tid=0 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 client.25522 shutdown 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 msgr2=0x556e515c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515c9340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0028005c40 tx=0x7f0028003d80 comp rx=0 tx=0).stop 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 shutdown_connections 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e515d9810 0x556e515dbc00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515c3fa0 0x556e515c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f00177fe640 0 client.25522 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 --2- 172.21.15.37:0/3216290806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515bfa10 0x556e515c9340 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 >> 172.21.15.37:0/3216290806 conn(0x556e515b1ec0 msgr2=0x556e515bf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 shutdown_connections 2024-04-26T18:29:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 1 -- 172.21.15.37:0/3216290806 wait complete. 2024-04-26T18:29:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.352+0000 7f004142d740 20 client.25522 trim_cache size 0 max 16384 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 Processor -- start 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 -- start start 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515d5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c3fa0 0x556e515cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00381aec00 con 0x556e515c03d0 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00381ac4d0 con 0x556e515bfa10 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.370+0000 7f004142d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00381aa320 con 0x556e515c3fa0 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59242/0 (socket says 172.21.15.37:59242) 2024-04-26T18:29:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 -- 172.21.15.37:0/872894723 learned_addr learned my addr 172.21.15.37:0/872894723 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f0037fff640 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515d5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f003ccb1640 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c3fa0 0x556e515cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 -- 172.21.15.37:0/872894723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c3fa0 msgr2=0x556e515cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c3fa0 0x556e515cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 -- 172.21.15.37:0/872894723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 msgr2=0x556e515d5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515d5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 -- 172.21.15.37:0/872894723 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e515c7890 con 0x556e515c03d0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f00377fe640 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0020099570 tx=0x7f00200d3120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa79db674283357b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f002dffb640 1 -- 172.21.15.37:0/872894723 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00200b6070 con 0x556e515c03d0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f002dffb640 1 -- 172.21.15.37:0/872894723 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002009d2a0 con 0x556e515c03d0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.371+0000 7f002dffb640 1 -- 172.21.15.37:0/872894723 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00200046c0 con 0x556e515c03d0 2024-04-26T18:29:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- 172.21.15.37:0/872894723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 msgr2=0x556e515c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0020099570 tx=0x7f00200d3120 comp rx=0 tx=0).stop 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- 172.21.15.37:0/872894723 shutdown_connections 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c3fa0 0x556e515cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c03d0 0x556e515c8c20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 --2- 172.21.15.37:0/872894723 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515d5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- 172.21.15.37:0/872894723 >> 172.21.15.37:0/872894723 conn(0x556e515b1ec0 msgr2=0x556e515cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- 172.21.15.37:0/872894723 shutdown_connections 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- 172.21.15.37:0/872894723 wait complete. 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.372+0000 7f004142d740 1 -- start start 2024-04-26T18:29:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 Processor -- start 2024-04-26T18:29:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c03d0 0x556e515e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 0x556e515e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00381aa990 con 0x556e515c3fa0 2024-04-26T18:29:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f00381abe00 con 0x556e515bfa10 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00381ab060 con 0x556e515c03d0 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f004142d740 10 client.0 _ll_register_callbacks cb 0x7f0028040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f003ccb1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 0x556e515e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f00377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c03d0 0x556e515e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:33058/0 (socket says 172.21.15.37:33058) 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 -- 172.21.15.37:0/3418557957 learned_addr learned my addr 172.21.15.37:0/3418557957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 -- 172.21.15.37:0/3418557957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c03d0 msgr2=0x556e515e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c03d0 0x556e515e0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 -- 172.21.15.37:0/3418557957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 msgr2=0x556e515e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 0x556e515e1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f0037fff640 1 -- 172.21.15.37:0/3418557957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5163ece0 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.373+0000 7f003ccb1640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 0x556e515e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f0037fff640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f002803f040 tx=0x7f0028003ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed9845239adee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028057030 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f002803c640 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f002009d2a0 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f002804e040 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f004142d740 10 client.25586 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e5163fff0 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002009d2a0 con 0x556e515bfa10 2024-04-26T18:29:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.374+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0028004770 con 0x556e515bfa10 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 10 client.25586 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 20 client.25586 populate_metadata read hostname 'smithi037' 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e51640650 0x556e51642a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 4 client.25586 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e51523ee0 con 0x556e51640650 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f00377fe640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e51640650 0x556e51642a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f00377fe640 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e51640650 0x556e51642a40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7450fbb5111d21 server_cookie=368dedc265b44be in_seq=0 out_seq=0 2024-04-26T18:29:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.375+0000 7f00177fe640 10 client.25586 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.392+0000 7f00177fe640 1 -- 172.21.15.37:0/3418557957 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e51523ee0 con 0x556e51640650 2024-04-26T18:29:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.392+0000 7f00177fe640 10 client.25586 handle_command_reply: tid=0 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.392+0000 7f004142d740 1 client.25586 shutdown 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 msgr2=0x556e515e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f002803f040 tx=0x7f0028003ab0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 shutdown_connections 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x556e51640650 0x556e51642a40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e515c3fa0 0x556e515e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556e515c03d0 0x556e515e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f00177fe640 0 client.25586 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 --2- 172.21.15.37:0/3418557957 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e515bfa10 0x556e515e0970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 >> 172.21.15.37:0/3418557957 conn(0x556e515b1ec0 msgr2=0x556e515bf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 shutdown_connections 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 1 -- 172.21.15.37:0/3418557957 wait complete. 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.393+0000 7f004142d740 20 client.25586 trim_cache size 0 max 16384 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:29:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:29:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 msgr2=0x7f00240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 0x7f00240a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0030009fd0 tx=0x7f0030004ab0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 msgr2=0x7f003819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f002000e9f0 tx=0x7f002000edf0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 shutdown_connections 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f00240a3dc0 0x7f00240a6280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0038107860 0x7f00381a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0038102210 0x7f003819a440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 --2- 172.21.15.37:0/4026309008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0038101850 0x7f0038199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 >> 172.21.15.37:0/4026309008 conn(0x7f0038077bb0 msgr2=0x7f00380ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 shutdown_connections 2024-04-26T18:29:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:29:51.394+0000 7f003e714640 1 -- 172.21.15.37:0/4026309008 wait complete. 2024-04-26T18:29:51.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:51 smithi037 ceph-mon[29171]: pgmap v168: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:51 smithi120 ceph-mon[32244]: pgmap v168: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:51 smithi170 ceph-mon[32697]: pgmap v168: 97 pgs: 97 active+clean; 97 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 934 op/s 2024-04-26T18:29:53.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:53 smithi037 ceph-mon[29171]: pgmap v169: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.8 MiB/s wr, 1.37k op/s 2024-04-26T18:29:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:53 smithi120 ceph-mon[32244]: pgmap v169: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.8 MiB/s wr, 1.37k op/s 2024-04-26T18:29:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:53 smithi170 ceph-mon[32697]: pgmap v169: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.8 MiB/s wr, 1.37k op/s 2024-04-26T18:29:55.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:55 smithi037 ceph-mon[29171]: pgmap v170: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 892 op/s 2024-04-26T18:29:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:55 smithi120 ceph-mon[32244]: pgmap v170: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 892 op/s 2024-04-26T18:29:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:55 smithi170 ceph-mon[32697]: pgmap v170: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 892 op/s 2024-04-26T18:29:57.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:57 smithi037 ceph-mon[29171]: pgmap v171: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.32k op/s 2024-04-26T18:29:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:57 smithi120 ceph-mon[32244]: pgmap v171: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.32k op/s 2024-04-26T18:29:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:57 smithi170 ceph-mon[32697]: pgmap v171: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.32k op/s 2024-04-26T18:29:59.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:29:59 smithi037 ceph-mon[29171]: pgmap v172: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:29:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:29:59 smithi120 ceph-mon[32244]: pgmap v172: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:29:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:29:59 smithi170 ceph-mon[32697]: pgmap v172: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:29:59.721 INFO:tasks.workunit.client.0.smithi037.stdout:copying to largest ever: 0x63ef9ee 2024-04-26T18:30:00.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:00 smithi037 ceph-mon[29171]: overall HEALTH_OK 2024-04-26T18:30:00.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:00 smithi120 ceph-mon[32244]: overall HEALTH_OK 2024-04-26T18:30:00.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:00 smithi170 ceph-mon[32697]: overall HEALTH_OK 2024-04-26T18:30:01.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:01 smithi037 ceph-mon[29171]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:30:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:01 smithi120 ceph-mon[32244]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:30:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:01 smithi170 ceph-mon[32697]: pgmap v173: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 868 op/s 2024-04-26T18:30:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:03 smithi037 ceph-mon[29171]: pgmap v174: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.28k op/s 2024-04-26T18:30:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:03 smithi120 ceph-mon[32244]: pgmap v174: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.28k op/s 2024-04-26T18:30:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:03 smithi170 ceph-mon[32697]: pgmap v174: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.28k op/s 2024-04-26T18:30:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 -- 172.21.15.37:0/2407554811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 msgr2=0x7f27c410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/2407554811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c410f770 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009900 tx=0x7f27b402a410 comp rx=0 tx=0).stop 2024-04-26T18:30:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 -- 172.21.15.37:0/2407554811 shutdown_connections 2024-04-26T18:30:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/2407554811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c406b160 0x7f27c406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/2407554811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c410f770 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/2407554811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c4108920 0x7f27c4108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 -- 172.21.15.37:0/2407554811 >> 172.21.15.37:0/2407554811 conn(0x7f27c4077630 msgr2=0x7f27c4077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 -- 172.21.15.37:0/2407554811 shutdown_connections 2024-04-26T18:30:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.499+0000 7f27c8fa4640 1 -- 172.21.15.37:0/2407554811 wait complete. 2024-04-26T18:30:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 Processor -- start 2024-04-26T18:30:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 -- start start 2024-04-26T18:30:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 0x7f27c419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 0x7f27c419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112bc0 con 0x7f27c406bb00 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112a40 con 0x7f27c4108920 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c8fa4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112d40 con 0x7f27c406b160 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 0x7f27c419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54040/0 (socket says 172.21.15.37:54040) 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 -- 172.21.15.37:0/191948148 learned_addr learned my addr 172.21.15.37:0/191948148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c259c640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 0x7f27c419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 -- 172.21.15.37:0/191948148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 msgr2=0x7f27c419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 0x7f27c419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 -- 172.21.15.37:0/191948148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 msgr2=0x7f27c419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 0x7f27c419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c1d9b640 1 -- 172.21.15.37:0/191948148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c41a0200 con 0x7f27c406bb00 2024-04-26T18:30:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 0x7f27c419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.500+0000 7f27c259c640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 0x7f27c419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e9f0 tx=0x7f27b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27b37fe640 1 -- 172.21.15.37:0/191948148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b800cd80 con 0x7f27c406bb00 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27b37fe640 1 -- 172.21.15.37:0/191948148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b80108e0 con 0x7f27c406bb00 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c41a04f0 con 0x7f27c406bb00 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27b37fe640 1 -- 172.21.15.37:0/191948148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8018980 con 0x7f27c406bb00 2024-04-26T18:30:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.501+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a7d80 con 0x7f27c406bb00 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27b37fe640 1 -- 172.21.15.37:0/191948148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27b8010430 con 0x7f27c406bb00 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 Processor -- start 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 -- start start 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x5573659432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x5573659504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573658a9460 con 0x557365943820 2024-04-26T18:30:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557365863590 con 0x55736594e0e0 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27cbd98740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5573658a5420 con 0x557365942f00 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54052/0 (socket says 172.21.15.37:54052) 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27c259c640 1 -- 172.21.15.37:0/3066551514 learned_addr learned my addr 172.21.15.37:0/3066551514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x5573659432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.504+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x5573659504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27b37fe640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 0x7f27ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 -- 172.21.15.37:0/3066551514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 msgr2=0x5573659432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x5573659432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 -- 172.21.15.37:0/3066551514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 msgr2=0x5573659504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x5573659504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 0x7f27ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 -- 172.21.15.37:0/3066551514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573658700d0 con 0x557365943820 2024-04-26T18:30:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27b37fe640 1 -- 172.21.15.37:0/191948148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f27b80d0b50 con 0x7f27c406bb00 2024-04-26T18:30:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c259c640 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009fd0 tx=0x7f27bc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c397c3c1330db1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 0x7f27ac0a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006010 tx=0x7f27b402a410 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27b0ff9640 1 -- 172.21.15.37:0/3066551514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc011760 con 0x557365943820 2024-04-26T18:30:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27b0ff9640 1 -- 172.21.15.37:0/3066551514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27bc011db0 con 0x557365943820 2024-04-26T18:30:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.505+0000 7f27b0ff9640 1 -- 172.21.15.37:0/3066551514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00d340 con 0x557365943820 2024-04-26T18:30:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 -- 172.21.15.37:0/3066551514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 msgr2=0x55736594dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009fd0 tx=0x7f27bc0108e0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 -- 172.21.15.37:0/3066551514 shutdown_connections 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x5573659504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x55736594dba0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3066551514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x5573659432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 -- 172.21.15.37:0/3066551514 >> 172.21.15.37:0/3066551514 conn(0x55736593f410 msgr2=0x55736593f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 -- 172.21.15.37:0/3066551514 shutdown_connections 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.506+0000 7f27cbd98740 1 -- 172.21.15.37:0/3066551514 wait complete. 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 -- start start 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 Processor -- start 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x55736594b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557365950d10 con 0x55736594e0e0 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557365830b60 con 0x557365943820 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557365950e90 con 0x557365942f00 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27cbd98740 10 client.0 _ll_register_callbacks cb 0x5573658552b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.507+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54066/0 (socket says 172.21.15.37:54066) 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:49172/0 (socket says 172.21.15.37:49172) 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c2d9d640 1 -- 172.21.15.37:0/3069950534 learned_addr learned my addr 172.21.15.37:0/3069950534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c259c640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x55736594b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 -- 172.21.15.37:0/3069950534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 msgr2=0x55736594b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x55736594b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 -- 172.21.15.37:0/3069950534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 msgr2=0x55736594baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 -- 172.21.15.37:0/3069950534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573655bb210 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c259c640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x55736594b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ce290 tx=0x7f27b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d513018c52f5af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b809d1e0 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f27b8010a90 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27bc011db0 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.508+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27b80a0e90 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 10 client.25598 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557365687620 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27bc011db0 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f27b8098830 con 0x557365943820 2024-04-26T18:30:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 10 client.25598 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 20 client.25598 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55736595e230 0x557365960620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 4 client.25598 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573658b7070 con 0x55736595e230 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.509+0000 7f27c259c640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55736595e230 0x557365960620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.510+0000 7f27a9ffb640 10 client.25598 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.510+0000 7f27c259c640 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55736595e230 0x557365960620 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898278c08383c1bd server_cookie=5bd41a9414eb1138 in_seq=0 out_seq=0 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.510+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3069950534 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573658b7070 con 0x55736595e230 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.510+0000 7f27a9ffb640 10 client.25598 handle_command_reply: tid=0 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.510+0000 7f27cbd98740 1 client.25598 shutdown 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 msgr2=0x55736594b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ce290 tx=0x7f27b8000bc0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 shutdown_connections 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55736595e230 0x557365960620 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x55736594b560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x55736594baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27a9ffb640 0 client.25598 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3069950534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x55736594b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 >> 172.21.15.37:0/3069950534 conn(0x55736593f410 msgr2=0x55736593e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 shutdown_connections 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 1 -- 172.21.15.37:0/3069950534 wait complete. 2024-04-26T18:30:04.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.511+0000 7f27cbd98740 20 client.25598 trim_cache size 0 max 16384 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 Processor -- start 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 -- start start 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365965ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x55736594c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b2fd0 con 0x557365943820 2024-04-26T18:30:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b08a0 con 0x55736594e0e0 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27cbd98740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ae6f0 con 0x557365942f00 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54080/0 (socket says 172.21.15.37:54080) 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 -- 172.21.15.37:0/2263932599 learned_addr learned my addr 172.21.15.37:0/2263932599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365965ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x55736594c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 -- 172.21.15.37:0/2263932599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 msgr2=0x557365965ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365965ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 -- 172.21.15.37:0/2263932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 msgr2=0x55736594c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x55736594c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 -- 172.21.15.37:0/2263932599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573659458c0 con 0x557365943820 2024-04-26T18:30:04.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.527+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f27b80d0da0 tx=0x7f27b8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2424f60d5c986f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27abfff640 1 -- 172.21.15.37:0/2263932599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b809c440 con 0x557365943820 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27abfff640 1 -- 172.21.15.37:0/2263932599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b8010720 con 0x557365943820 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27abfff640 1 -- 172.21.15.37:0/2263932599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b80ce510 con 0x557365943820 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 -- 172.21.15.37:0/2263932599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 msgr2=0x557365966420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f27b80d0da0 tx=0x7f27b8018420 comp rx=0 tx=0).stop 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 -- 172.21.15.37:0/2263932599 shutdown_connections 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55736594e0e0 0x55736594c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557365943820 0x557365966420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 --2- 172.21.15.37:0/2263932599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365965ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 -- 172.21.15.37:0/2263932599 >> 172.21.15.37:0/2263932599 conn(0x55736593f410 msgr2=0x557365946360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 -- 172.21.15.37:0/2263932599 shutdown_connections 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.528+0000 7f27cbd98740 1 -- 172.21.15.37:0/2263932599 wait complete. 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 -- start start 2024-04-26T18:30:04.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 Processor -- start 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365948430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x557365948970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41aed60 con 0x55736594e0e0 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41b01d0 con 0x557365943820 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41af430 con 0x557365942f00 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27cbd98740 10 client.0 _ll_register_callbacks cb 0x7f27b8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365948430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54084/0 (socket says 172.21.15.37:54084) 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 -- 172.21.15.37:0/3427690325 learned_addr learned my addr 172.21.15.37:0/3427690325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x557365948970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 -- 172.21.15.37:0/3427690325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 msgr2=0x557365948430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365948430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 -- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 msgr2=0x557365948970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x557365948970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c2d9d640 1 -- 172.21.15.37:0/3427690325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573659c3c90 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.529+0000 7f27c1d9b640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x557365948970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27c2d9d640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f27b4031440 tx=0x7f27b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cd37a150ca1133 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4035040 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f27b4004700 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573658b0ee0 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27b4005610 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 10 client.15990 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573659c4f50 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5573658b0ee0 con 0x55736594e0e0 2024-04-26T18:30:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f27b4003a90 con 0x55736594e0e0 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 20 client.15990 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5573659c5680 0x5573659c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 4 client.15990 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.530+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55736582c5f0 con 0x5573659c5680 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.531+0000 7f27c259c640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5573659c5680 0x5573659c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.531+0000 7f27a9ffb640 10 client.15990 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.531+0000 7f27c259c640 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5573659c5680 0x5573659c7aa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c680e71d57c26d server_cookie=cfa000bbe4a3cfce in_seq=0 out_seq=0 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27a9ffb640 1 -- 172.21.15.37:0/3427690325 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55736582c5f0 con 0x5573659c5680 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27a9ffb640 10 client.15990 handle_command_reply: tid=0 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 client.15990 shutdown 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 msgr2=0x5573659c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f27b4031440 tx=0x7f27b4009260 comp rx=0 tx=0).stop 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 shutdown_connections 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x5573659c5680 0x5573659c7aa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557365943820 0x557365948970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55736594e0e0 0x5573659c3750 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 --2- 172.21.15.37:0/3427690325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557365942f00 0x557365948430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 >> 172.21.15.37:0/3427690325 conn(0x55736593f410 msgr2=0x55736593dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27a9ffb640 0 client.15990 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 shutdown_connections 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.549+0000 7f27cbd98740 1 -- 172.21.15.37:0/3427690325 wait complete. 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27cbd98740 20 client.15990 trim_cache size 0 max 16384 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:30:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 msgr2=0x7f27ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 0x7f27ac0a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006010 tx=0x7f27b402a410 comp rx=0 tx=0).stop 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 msgr2=0x7f27c419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e9f0 tx=0x7f27b800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 shutdown_connections 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27ac0a3e40 0x7f27ac0a6300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27c4108920 0x7f27c419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27c406bb00 0x7f27c419b750 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 --2- 172.21.15.37:0/191948148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27c406b160 0x7f27c419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 >> 172.21.15.37:0/191948148 conn(0x7f27c4077630 msgr2=0x7f27c410dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 shutdown_connections 2024-04-26T18:30:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:04.550+0000 7f27c8fa4640 1 -- 172.21.15.37:0/191948148 wait complete. 2024-04-26T18:30:05.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:05 smithi037 ceph-mon[29171]: pgmap v175: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 842 op/s 2024-04-26T18:30:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:05 smithi120 ceph-mon[32244]: pgmap v175: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 842 op/s 2024-04-26T18:30:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:05 smithi170 ceph-mon[32697]: pgmap v175: 97 pgs: 97 active+clean; 107 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 842 op/s 2024-04-26T18:30:07.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:07 smithi037 ceph-mon[29171]: pgmap v176: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.27k op/s 2024-04-26T18:30:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:07 smithi120 ceph-mon[32244]: pgmap v176: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.27k op/s 2024-04-26T18:30:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:07 smithi170 ceph-mon[32697]: pgmap v176: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.27k op/s 2024-04-26T18:30:09.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:09 smithi037 ceph-mon[29171]: pgmap v177: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:09 smithi120 ceph-mon[32244]: pgmap v177: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:09 smithi170 ceph-mon[32697]: pgmap v177: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:11 smithi037 ceph-mon[29171]: pgmap v178: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:11 smithi120 ceph-mon[32244]: pgmap v178: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:11 smithi170 ceph-mon[32697]: pgmap v178: 97 pgs: 97 active+clean; 111 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 839 op/s 2024-04-26T18:30:13.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:13 smithi037 ceph-mon[29171]: pgmap v179: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 1.24k op/s 2024-04-26T18:30:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:13 smithi120 ceph-mon[32244]: pgmap v179: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 1.24k op/s 2024-04-26T18:30:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:13 smithi170 ceph-mon[32697]: pgmap v179: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 1.24k op/s 2024-04-26T18:30:15.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:15 smithi037 ceph-mon[29171]: pgmap v180: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 832 op/s 2024-04-26T18:30:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:15 smithi120 ceph-mon[32244]: pgmap v180: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 832 op/s 2024-04-26T18:30:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:15 smithi170 ceph-mon[32697]: pgmap v180: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 832 op/s 2024-04-26T18:30:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:17 smithi037 ceph-mon[29171]: pgmap v181: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.23k op/s 2024-04-26T18:30:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:17 smithi120 ceph-mon[32244]: pgmap v181: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.23k op/s 2024-04-26T18:30:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:17 smithi170 ceph-mon[32697]: pgmap v181: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.23k op/s 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2906895677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc101810 msgr2=0x7fa3dc101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2906895677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc101810 0x7fa3dc101c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009cd0 tx=0x7fa3cc02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2906895677 shutdown_connections 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2906895677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2906895677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc101810 0x7fa3dc101c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2906895677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc107820 0x7fa3dc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2906895677 >> 172.21.15.37:0/2906895677 conn(0x7fa3dc0fd300 msgr2=0x7fa3dc0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2906895677 shutdown_connections 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.654+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2906895677 wait complete. 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 Processor -- start 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 -- start start 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 0x7fa3dc1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc111b30 con 0x7fa3dc101810 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1119b0 con 0x7fa3dc1021d0 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e2c6b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc111cb0 con 0x7fa3dc107820 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e0a07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 0x7fa3dc1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.37:40550/0 (socket says 172.21.15.37:40550) 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:58710/0 (socket says 172.21.15.37:58710) 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3dbfff640 1 -- 172.21.15.37:0/2877882091 learned_addr learned my addr 172.21.15.37:0/2877882091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.655+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 0x7fa3dc1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44622/0 (socket says 172.21.15.37:44622) 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 -- 172.21.15.37:0/2877882091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 msgr2=0x7fa3dc19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 -- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 msgr2=0x7fa3dc1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 0x7fa3dc1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 -- 172.21.15.37:0/2877882091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc19e0d0 con 0x7fa3dc1021d0 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3e1208640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005e30 tx=0x7fa3cc0039f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3d9ffb640 1 -- 172.21.15.37:0/2877882091 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc004040 con 0x7fa3dc1021d0 2024-04-26T18:30:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc19e360 con 0x7fa3dc1021d0 2024-04-26T18:30:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3d9ffb640 1 -- 172.21.15.37:0/2877882091 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc004690 con 0x7fa3dc1021d0 2024-04-26T18:30:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3d9ffb640 1 -- 172.21.15.37:0/2877882091 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc046480 con 0x7fa3dc1021d0 2024-04-26T18:30:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.656+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc1a5c50 con 0x7fa3dc1021d0 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 Processor -- start 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 -- start start 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3d9ffb640 1 -- 172.21.15.37:0/2877882091 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3cc030630 con 0x7fa3dc1021d0 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933e8b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560933df1460 con 0x560933e8b700 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560933dab590 con 0x560933e9a4f0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e5989740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560933ded420 con 0x560933e8ade0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e0a07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e0a07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44624/0 (socket says 172.21.15.37:44624) 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e0a07640 1 -- 172.21.15.37:0/3827418705 learned_addr learned my addr 172.21.15.37:0/3827418705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3e1208640 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933e8b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 -- 172.21.15.37:0/3827418705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 msgr2=0x560933e8b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933e8b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 -- 172.21.15.37:0/3827418705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 msgr2=0x560933e99fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.660+0000 7fa3dbfff640 1 -- 172.21.15.37:0/3827418705 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560933db80d0 con 0x560933e9a4f0 2024-04-26T18:30:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3d9ffb640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 0x7fa3c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e9c8e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ef500 tx=0x7fa3cc0fa220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a236d1eef206f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3e1208640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 0x7fa3c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3d2ffd640 1 -- 172.21.15.37:0/3827418705 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b9ff0 con 0x560933e9a4f0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3d2ffd640 1 -- 172.21.15.37:0/3827418705 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0f7050 con 0x560933e9a4f0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3d2ffd640 1 -- 172.21.15.37:0/3827418705 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc087b70 con 0x560933e9a4f0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.661+0000 7fa3d9ffb640 1 -- 172.21.15.37:0/2877882091 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa3cc02e030 con 0x7fa3dc1021d0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- 172.21.15.37:0/3827418705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 msgr2=0x560933e9c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e9c8e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ef500 tx=0x7fa3cc0fa220 comp rx=0 tx=0).stop 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e1208640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 0x7fa3c80a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4012640 tx=0x7fa3d4005f90 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- 172.21.15.37:0/3827418705 shutdown_connections 2024-04-26T18:30:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e9c8e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e99fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 --2- 172.21.15.37:0/3827418705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933e8b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- 172.21.15.37:0/3827418705 >> 172.21.15.37:0/3827418705 conn(0x560933e87410 msgr2=0x560933e877f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- 172.21.15.37:0/3827418705 shutdown_connections 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- 172.21.15.37:0/3827418705 wait complete. 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 -- start start 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.662+0000 7fa3e5989740 1 Processor -- start 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 0x560933e97370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933e978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560933e9cee0 con 0x560933e8ade0 2024-04-26T18:30:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560933d78b60 con 0x560933e8b700 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560933e9d060 con 0x560933e9a4f0 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e5989740 10 client.0 _ll_register_callbacks cb 0x560933d9d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 0x560933e97370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.37:58736/0 (socket says 172.21.15.37:58736) 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e1208640 1 -- 172.21.15.37:0/929306371 learned_addr learned my addr 172.21.15.37:0/929306371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933e978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 -- 172.21.15.37:0/929306371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 msgr2=0x560933e97df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 -- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 msgr2=0x560933e978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933e978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e0a07640 1 -- 172.21.15.37:0/929306371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560933b03210 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933e978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.663+0000 7fa3e1208640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 0x560933e97370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4012680 tx=0x7fa3c4012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43edd612574c637a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4004050 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa3c40041f0 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc0f7050 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3c40043f0 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3e5989740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560933bcf620 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0f7050 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa3c4003b90 con 0x560933e8ade0 2024-04-26T18:30:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.664+0000 7fa3e5989740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3e5989740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3e5989740 20 client.16002 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933eaa330 0x560933eac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3e5989740 4 client.16002 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560933dff070 con 0x560933eaa330 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.665+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933eaa330 0x560933eac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933eaa330 0x560933eac720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61aac2ae9cfd3e5e server_cookie=5fb3f4acc03b2d2 in_seq=0 out_seq=0 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3bbfff640 10 client.16002 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3bbfff640 1 -- 172.21.15.37:0/929306371 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560933dff070 con 0x560933eaa330 2024-04-26T18:30:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3bbfff640 10 client.16002 handle_command_reply: tid=0 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 client.16002 shutdown 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 msgr2=0x560933e97370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 0x560933e97370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4012680 tx=0x7fa3c4012b40 comp rx=0 tx=0).stop 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 shutdown_connections 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933eaa330 0x560933eac720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933e978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e9a4f0 0x560933e97df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3bbfff640 0 client.16002 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 --2- 172.21.15.37:0/929306371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8ade0 0x560933e97370 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.666+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 >> 172.21.15.37:0/929306371 conn(0x560933e87410 msgr2=0x560933e8d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.667+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 shutdown_connections 2024-04-26T18:30:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.667+0000 7fa3e5989740 1 -- 172.21.15.37:0/929306371 wait complete. 2024-04-26T18:30:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.667+0000 7fa3e5989740 20 client.16002 trim_cache size 0 max 16384 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 Processor -- start 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 -- start start 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.683+0000 7fa3e5989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1aec80 con 0x560933e8b700 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3e5989740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1ac550 con 0x560933e9a4f0 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3e5989740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1aa3a0 con 0x560933e8ade0 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44644/0 (socket says 172.21.15.37:44644) 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 -- 172.21.15.37:0/213299839 learned_addr learned my addr 172.21.15.37:0/213299839 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3e1208640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 -- 172.21.15.37:0/213299839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 msgr2=0x560933ea5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 -- 172.21.15.37:0/213299839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 msgr2=0x560933e865b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e865b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 -- 172.21.15.37:0/213299839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560933e8b530 con 0x560933e8b700 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3e1208640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0bda50 tx=0x7fa3cc0309d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95caf89929e4ce11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3d27fc640 1 -- 172.21.15.37:0/213299839 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc087d50 con 0x560933e8b700 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3d27fc640 1 -- 172.21.15.37:0/213299839 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0f7040 con 0x560933e8b700 2024-04-26T18:30:17.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.684+0000 7fa3d27fc640 1 -- 172.21.15.37:0/213299839 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc087fe0 con 0x560933e8b700 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- 172.21.15.37:0/213299839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 msgr2=0x560933e96c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0bda50 tx=0x7fa3cc0309d0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- 172.21.15.37:0/213299839 shutdown_connections 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e9a4f0 0x560933e865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e8b700 0x560933e96c70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 --2- 172.21.15.37:0/213299839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- 172.21.15.37:0/213299839 >> 172.21.15.37:0/213299839 conn(0x560933e87410 msgr2=0x560933e98ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- 172.21.15.37:0/213299839 shutdown_connections 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- 172.21.15.37:0/213299839 wait complete. 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.685+0000 7fa3e5989740 1 -- start start 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 Processor -- start 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933ea2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1aaa10 con 0x560933e9a4f0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1abe80 con 0x560933e8b700 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc1ab0e0 con 0x560933e8ade0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e5989740 10 client.0 _ll_register_callbacks cb 0x7fa3c4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44648/0 (socket says 172.21.15.37:44648) 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 -- 172.21.15.37:0/82713308 learned_addr learned my addr 172.21.15.37:0/82713308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3dbfff640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933ea2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 -- 172.21.15.37:0/82713308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 msgr2=0x560933ea2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 -- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 msgr2=0x560933ea2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.686+0000 7fa3e1208640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933ea2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e1208640 1 -- 172.21.15.37:0/82713308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560933e94070 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e1208640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009f70 tx=0x7fa3d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a94e50ce04d4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400fde0 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7fa3d4005020 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3cc0f7040 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3d4005220 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e5989740 10 client.16008 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560933f10250 con 0x560933e9a4f0 2024-04-26T18:30:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.687+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc0f7040 con 0x560933e9a4f0 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa3d402a400 con 0x560933e9a4f0 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 20 client.16008 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933f10a50 0x560933f12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 4 client.16008 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560933df8ee0 con 0x560933f10a50 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933f10a50 0x560933f12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3bbfff640 10 client.16008 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:17.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.688+0000 7fa3e0a07640 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933f10a50 0x560933f12e70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccad8da479a9817d server_cookie=c3a10f4cf7662c6a in_seq=0 out_seq=0 2024-04-26T18:30:17.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.700+0000 7fa3bbfff640 1 -- 172.21.15.37:0/82713308 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560933df8ee0 con 0x560933f10a50 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3bbfff640 10 client.16008 handle_command_reply: tid=0 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 client.16008 shutdown 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 msgr2=0x560933e939d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009f70 tx=0x7fa3d40046c0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 shutdown_connections 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x560933f10a50 0x560933f12e70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3bbfff640 0 client.16008 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560933e8b700 0x560933ea2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560933e9a4f0 0x560933e939d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 --2- 172.21.15.37:0/82713308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560933e8ade0 0x560933ea2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 >> 172.21.15.37:0/82713308 conn(0x560933e87410 msgr2=0x560933e99430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 shutdown_connections 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 1 -- 172.21.15.37:0/82713308 wait complete. 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.701+0000 7fa3e5989740 20 client.16008 trim_cache size 0 max 16384 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:30:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:30:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 msgr2=0x7fa3c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 0x7fa3c80a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4012640 tx=0x7fa3d4005f90 comp rx=0 tx=0).stop 2024-04-26T18:30:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 msgr2=0x7fa3dc199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005e30 tx=0x7fa3cc0039f0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 shutdown_connections 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fa3c80a3dc0 0x7fa3c80a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3dc1021d0 0x7fa3dc199620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3dc107820 0x7fa3dc19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 --2- 172.21.15.37:0/2877882091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc101810 0x7fa3dc1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 >> 172.21.15.37:0/2877882091 conn(0x7fa3dc0fd300 msgr2=0x7fa3dc06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 shutdown_connections 2024-04-26T18:30:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:17.702+0000 7fa3e2c6b640 1 -- 172.21.15.37:0/2877882091 wait complete. 2024-04-26T18:30:19.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:19 smithi037 ceph-mon[29171]: pgmap v182: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:19 smithi120 ceph-mon[32244]: pgmap v182: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:19 smithi170 ceph-mon[32697]: pgmap v182: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:21.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:21 smithi037 ceph-mon[29171]: pgmap v183: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:21 smithi120 ceph-mon[32244]: pgmap v183: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:21 smithi170 ceph-mon[32697]: pgmap v183: 97 pgs: 97 active+clean; 118 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 805 op/s 2024-04-26T18:30:22.445 INFO:tasks.workunit.client.0.smithi037.stdout:All 50000 operations completed A-OK! 2024-04-26T18:30:22.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T18:30:22.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T18:30:23.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:23 smithi037 ceph-mon[29171]: pgmap v184: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2024-04-26T18:30:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:23 smithi120 ceph-mon[32244]: pgmap v184: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2024-04-26T18:30:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:23 smithi170 ceph-mon[32697]: pgmap v184: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.19k op/s 2024-04-26T18:30:25.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:25 smithi037 ceph-mon[29171]: pgmap v185: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-04-26T18:30:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:25 smithi120 ceph-mon[32244]: pgmap v185: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-04-26T18:30:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:25 smithi170 ceph-mon[32697]: pgmap v185: 97 pgs: 97 active+clean; 104 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 790 op/s 2024-04-26T18:30:27.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:27 smithi037 ceph-mon[29171]: pgmap v186: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.2 MiB/s wr, 975 op/s 2024-04-26T18:30:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:27 smithi120 ceph-mon[32244]: pgmap v186: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.2 MiB/s wr, 975 op/s 2024-04-26T18:30:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:27 smithi170 ceph-mon[32697]: pgmap v186: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.2 MiB/s wr, 975 op/s 2024-04-26T18:30:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:29 smithi037 ceph-mon[29171]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:29 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:29 smithi120 ceph-mon[32244]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:29 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:29 smithi170 ceph-mon[32697]: pgmap v187: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:29 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 -- 172.21.15.37:0/2108097900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b01044e0 msgr2=0x7f43b010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 --2- 172.21.15.37:0/2108097900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b01044e0 0x7f43b010c9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009900 tx=0x7f43a402a510 comp rx=0 tx=0).stop 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 -- 172.21.15.37:0/2108097900 shutdown_connections 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 --2- 172.21.15.37:0/2108097900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b010cf00 0x7f43b010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 --2- 172.21.15.37:0/2108097900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b01044e0 0x7f43b010c9c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 --2- 172.21.15.37:0/2108097900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 0x7f43b0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 -- 172.21.15.37:0/2108097900 >> 172.21.15.37:0/2108097900 conn(0x7f43b00789b0 msgr2=0x7f43b0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 -- 172.21.15.37:0/2108097900 shutdown_connections 2024-04-26T18:30:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.797+0000 7f43b650a640 1 -- 172.21.15.37:0/2108097900 wait complete. 2024-04-26T18:30:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 Processor -- start 2024-04-26T18:30:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 -- start start 2024-04-26T18:30:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 0x7f43b01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b01044e0 0x7f43b01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0112bb0 con 0x7f43b010cf00 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0112a30 con 0x7f43b0103bc0 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b650a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0112d30 con 0x7f43b01044e0 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b4aa7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b01044e0 0x7f43b01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b4aa7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33350/0 (socket says 172.21.15.37:33350) 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.798+0000 7f43b4aa7640 1 -- 172.21.15.37:0/3514387907 learned_addr learned my addr 172.21.15.37:0/3514387907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43affff640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 0x7f43b01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 -- 172.21.15.37:0/3514387907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b01044e0 msgr2=0x7f43b01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b01044e0 0x7f43b01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 -- 172.21.15.37:0/3514387907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 msgr2=0x7f43b01990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 0x7f43b01990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 -- 172.21.15.37:0/3514387907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b019e0a0 con 0x7f43b010cf00 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b4aa7640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f43a800eb10 tx=0x7f43a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43ad7fa640 1 -- 172.21.15.37:0/3514387907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004b20 con 0x7f43b010cf00 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43ad7fa640 1 -- 172.21.15.37:0/3514387907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8004cc0 con 0x7f43b010cf00 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b019e390 con 0x7f43b010cf00 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43ad7fa640 1 -- 172.21.15.37:0/3514387907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8010670 con 0x7f43b010cf00 2024-04-26T18:30:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.799+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01a5c20 con 0x7f43b010cf00 2024-04-26T18:30:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.801+0000 7f43ad7fa640 1 -- 172.21.15.37:0/3514387907 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43a8005020 con 0x7f43b010cf00 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.801+0000 7f43b9225740 1 Processor -- start 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.801+0000 7f43b9225740 1 -- start start 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 0x55c37ae79fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c37add1460 con 0x55c37ae7a4f0 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c37ad8b590 con 0x55c37ae6b720 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b9225740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c37adcd420 con 0x55c37ae6ae00 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43ad7fa640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 0x7f43980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 0x55c37ae79fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae7c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33358/0 (socket says 172.21.15.37:33358) 2024-04-26T18:30:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43af7fe640 1 -- 172.21.15.37:0/638088842 learned_addr learned my addr 172.21.15.37:0/638088842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43af7fe640 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43b4aa7640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 0x7f43980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43ad7fa640 1 -- 172.21.15.37:0/3514387907 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f43a80d0d70 con 0x7f43b010cf00 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 -- 172.21.15.37:0/638088842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 msgr2=0x55c37ae6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae6b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 -- 172.21.15.37:0/638088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 msgr2=0x55c37ae7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 -- 172.21.15.37:0/638088842 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37ad980d0 con 0x55c37ae6b720 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43af7fe640 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.802+0000 7f43affff640 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 0x55c37ae79fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009b70 tx=0x7f43a00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=625a3136154288bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b4aa7640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 0x7f43980a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43a809ca00 tx=0x7f43a800efd0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43967fc640 1 -- 172.21.15.37:0/638088842 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0009e60 con 0x55c37ae6b720 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43967fc640 1 -- 172.21.15.37:0/638088842 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a0012900 con 0x55c37ae6b720 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43967fc640 1 -- 172.21.15.37:0/638088842 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a000f9a0 con 0x55c37ae6b720 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 -- 172.21.15.37:0/638088842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 msgr2=0x55c37ae79fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 0x55c37ae79fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009b70 tx=0x7f43a00117b0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 -- 172.21.15.37:0/638088842 shutdown_connections 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6b720 0x55c37ae79fb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 --2- 172.21.15.37:0/638088842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 -- 172.21.15.37:0/638088842 >> 172.21.15.37:0/638088842 conn(0x55c37ae67410 msgr2=0x55c37ae677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 -- 172.21.15.37:0/638088842 shutdown_connections 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.803+0000 7f43b9225740 1 -- 172.21.15.37:0/638088842 wait complete. 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 -- start start 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 Processor -- start 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6ae00 0x55c37ae773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae77900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c37ae7d120 con 0x55c37ae7a4f0 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c37ad58b60 con 0x55c37ae6ae00 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c37ae7d2a0 con 0x55c37ae6b720 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43b9225740 10 client.0 _ll_register_callbacks cb 0x55c37ad7d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33362/0 (socket says 172.21.15.37:33362) 2024-04-26T18:30:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43af7fe640 1 -- 172.21.15.37:0/3681615519 learned_addr learned my addr 172.21.15.37:0/3681615519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43affff640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae77900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.804+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6ae00 0x55c37ae773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 -- 172.21.15.37:0/3681615519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 msgr2=0x55c37ae77900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae77900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 -- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6ae00 msgr2=0x55c37ae773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6ae00 0x55c37ae773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 -- 172.21.15.37:0/3681615519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37aae3210 con 0x55c37ae7a4f0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f43a4031440 tx=0x7f43a4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe171b0ac58848f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4035070 con 0x55c37ae7a4f0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f43a4004030 con 0x55c37ae7a4f0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a0012900 con 0x55c37ae7a4f0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43a4031900 con 0x55c37ae7a4f0 2024-04-26T18:30:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43b9225740 10 client.16020 fetch_fsmap learned FSMap version 30 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c37abaf620 con 0x55c37ae7a4f0 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a0012900 con 0x55c37ae7a4f0 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.805+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f43a403d450 con 0x55c37ae7a4f0 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 10 client.16020 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 20 client.16020 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37ae8a620 0x55c37ae8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 4 client.16020 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c37addf070 con 0x55c37ae8a620 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43affff640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37ae8a620 0x55c37ae8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f43affff640 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37ae8a620 0x55c37ae8ca10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3c903b65517b93 server_cookie=d140986a0aadb3c5 in_seq=0 out_seq=0 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.806+0000 7f438f7fe640 10 client.16020 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f438f7fe640 1 -- 172.21.15.37:0/3681615519 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c37addf070 con 0x55c37ae8a620 2024-04-26T18:30:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f438f7fe640 10 client.16020 handle_command_reply: tid=0 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 client.16020 shutdown 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 msgr2=0x55c37ae77e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f43a4031440 tx=0x7f43a4041040 comp rx=0 tx=0).stop 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 shutdown_connections 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37ae8a620 0x55c37ae8ca10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae7a4f0 0x55c37ae77e40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae77900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f438f7fe640 0 client.16020 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 --2- 172.21.15.37:0/3681615519 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae6ae00 0x55c37ae773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 >> 172.21.15.37:0/3681615519 conn(0x55c37ae67410 msgr2=0x55c37ae6d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 shutdown_connections 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 1 -- 172.21.15.37:0/3681615519 wait complete. 2024-04-26T18:30:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.807+0000 7f43b9225740 20 client.16020 trim_cache size 0 max 16384 2024-04-26T18:30:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 Processor -- start 2024-04-26T18:30:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 -- start start 2024-04-26T18:30:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01aed20 con 0x55c37ae6ae00 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01ac5c0 con 0x55c37ae7a4f0 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.822+0000 7f43b9225740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01aa410 con 0x55c37ae6b720 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33370/0 (socket says 172.21.15.37:33370) 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 -- 172.21.15.37:0/2931241337 learned_addr learned my addr 172.21.15.37:0/2931241337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43affff640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 -- 172.21.15.37:0/2931241337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 msgr2=0x55c37ae92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 -- 172.21.15.37:0/2931241337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 msgr2=0x55c37ae927a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae927a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 -- 172.21.15.37:0/2931241337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37ae67240 con 0x55c37ae6ae00 2024-04-26T18:30:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43affff640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f43af7fe640 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006010 tx=0x7f43a4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8211e34ec0b4881 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f4395ffb640 1 -- 172.21.15.37:0/2931241337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4058070 con 0x55c37ae6ae00 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f4395ffb640 1 -- 172.21.15.37:0/2931241337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a4032da0 con 0x55c37ae6ae00 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.823+0000 7f4395ffb640 1 -- 172.21.15.37:0/2931241337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4035070 con 0x55c37ae6ae00 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- 172.21.15.37:0/2931241337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 msgr2=0x55c37ae86020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f43a4006010 tx=0x7f43a4005b50 comp rx=0 tx=0).stop 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- 172.21.15.37:0/2931241337 shutdown_connections 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae927a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6b720 0x55c37ae92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 --2- 172.21.15.37:0/2931241337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6ae00 0x55c37ae86020 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- 172.21.15.37:0/2931241337 >> 172.21.15.37:0/2931241337 conn(0x55c37ae67410 msgr2=0x55c37ae677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- 172.21.15.37:0/2931241337 shutdown_connections 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- 172.21.15.37:0/2931241337 wait complete. 2024-04-26T18:30:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 -- start start 2024-04-26T18:30:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.824+0000 7f43b9225740 1 Processor -- start 2024-04-26T18:30:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae74d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01aaa80 con 0x55c37ae6b720 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01abef0 con 0x55c37ae7a4f0 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01ab150 con 0x55c37ae6ae00 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 10 client.0 _ll_register_callbacks cb 0x7f43a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33384/0 (socket says 172.21.15.37:33384) 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 -- 172.21.15.37:0/3834546413 learned_addr learned my addr 172.21.15.37:0/3834546413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae74d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 -- 172.21.15.37:0/3834546413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 msgr2=0x55c37ae742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 -- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 msgr2=0x55c37ae74d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae74d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 -- 172.21.15.37:0/3834546413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37aeefcd0 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae74d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43affff640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009b70 tx=0x7f43a000f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a488c5b88cb946cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f438f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0032060 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a4032da0 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.825+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f43a0005450 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43a000f690 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 10 client.16026 fetch_fsmap learned FSMap version 30 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c37aef0fc0 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a4032da0 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f43a002c030 con 0x55c37ae6b720 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 20 client.16026 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37aef15e0 0x55c37aef3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 4 client.16026 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c37add8ee0 con 0x55c37aef15e0 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.826+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37aef15e0 0x55c37aef3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.827+0000 7f43af7fe640 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37aef15e0 0x55c37aef3a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39abb39f88b81c6 server_cookie=7ba1815361dee3b2 in_seq=0 out_seq=0 2024-04-26T18:30:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.827+0000 7f438f7fe640 10 client.16026 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.842+0000 7f438f7fe640 1 -- 172.21.15.37:0/3834546413 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c37add8ee0 con 0x55c37aef15e0 2024-04-26T18:30:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.842+0000 7f438f7fe640 10 client.16026 handle_command_reply: tid=0 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.842+0000 7f43b9225740 1 client.16026 shutdown 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 msgr2=0x55c37ae74800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009b70 tx=0x7f43a000f4f0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 shutdown_connections 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x55c37aef15e0 0x55c37aef3a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c37ae7a4f0 0x55c37ae74d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c37ae6b720 0x55c37ae74800 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f438f7fe640 0 client.16026 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 --2- 172.21.15.37:0/3834546413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c37ae6ae00 0x55c37ae742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 >> 172.21.15.37:0/3834546413 conn(0x55c37ae67410 msgr2=0x55c37ae65d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 shutdown_connections 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 1 -- 172.21.15.37:0/3834546413 wait complete. 2024-04-26T18:30:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b9225740 20 client.16026 trim_cache size 0 max 16384 2024-04-26T18:30:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:30:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:30:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:30:30.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 msgr2=0x7f43980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 0x7f43980a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43a809ca00 tx=0x7f43a800efd0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 msgr2=0x7f43b019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.843+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f43a800eb10 tx=0x7f43a800c2d0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 shutdown_connections 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f43980a3dc0 0x7f43980a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43b010cf00 0x7f43b019d970 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43b01044e0 0x7f43b01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 --2- 172.21.15.37:0/3514387907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43b0103bc0 0x7f43b01990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 >> 172.21.15.37:0/3514387907 conn(0x7f43b00789b0 msgr2=0x7f43b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 shutdown_connections 2024-04-26T18:30:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:30.844+0000 7f43b650a640 1 -- 172.21.15.37:0/3514387907 wait complete. 2024-04-26T18:30:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:31 smithi037 ceph-mon[29171]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:31 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:31 smithi120 ceph-mon[32244]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:31 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:31 smithi170 ceph-mon[32697]: pgmap v188: 97 pgs: 97 active+clean; 117 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2024-04-26T18:30:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:31 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:32.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:32 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:32.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:32 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:32.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:32 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:30:32.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:32 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:32.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:32 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:32.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:32 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:30:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:32 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:32 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:32 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:30:33.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:33 smithi037 ceph-mon[29171]: pgmap v189: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 728 op/s 2024-04-26T18:30:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:33 smithi120 ceph-mon[32244]: pgmap v189: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 728 op/s 2024-04-26T18:30:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:33 smithi170 ceph-mon[32697]: pgmap v189: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 728 op/s 2024-04-26T18:30:35.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:35 smithi037 ceph-mon[29171]: pgmap v190: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 339 op/s 2024-04-26T18:30:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:35 smithi120 ceph-mon[32244]: pgmap v190: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 339 op/s 2024-04-26T18:30:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:35 smithi170 ceph-mon[32697]: pgmap v190: 97 pgs: 97 active+clean; 127 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 339 op/s 2024-04-26T18:30:37.202 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-26T18:30:37.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T18:30:37.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:37 smithi037 ceph-mon[29171]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 474 op/s 2024-04-26T18:30:37.489 DEBUG:teuthology.parallel:result is None 2024-04-26T18:30:37.489 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T18:30:37.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:37 smithi120 ceph-mon[32244]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 474 op/s 2024-04-26T18:30:37.523 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T18:30:37.523 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T18:30:37.534 INFO:tasks.quiescer:joining Quiescers 2024-04-26T18:30:37.534 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T18:30:37.534 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:30:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:37 smithi170 ceph-mon[32697]: pgmap v191: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 474 op/s 2024-04-26T18:30:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:30:39.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:39 smithi037 ceph-mon[29171]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 -- 172.21.15.37:0/485423048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 msgr2=0x7ff55010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 --2- 172.21.15.37:0/485423048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55010f1d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b0a0 tx=0x7ff54800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 -- 172.21.15.37:0/485423048 shutdown_connections 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 --2- 172.21.15.37:0/485423048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55010f1d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 --2- 172.21.15.37:0/485423048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550108390 0x7ff550108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 --2- 172.21.15.37:0/485423048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550102320 0x7ff5501027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 -- 172.21.15.37:0/485423048 >> 172.21.15.37:0/485423048 conn(0x7ff5500792d0 msgr2=0x7ff5500796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 -- 172.21.15.37:0/485423048 shutdown_connections 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.490+0000 7ff55800c700 1 -- 172.21.15.37:0/485423048 wait complete. 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 Processor -- start 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 -- start start 2024-04-26T18:30:39.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 0x7ff55019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 0x7ff5501997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5501124b0 con 0x7ff550102ce0 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff550112330 con 0x7ff550102320 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff55800c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff550112630 con 0x7ff550108390 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff5555a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff5555a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37626/0 (socket says 172.21.15.37:37626) 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff5555a7700 1 -- 172.21.15.37:0/2209506975 learned_addr learned my addr 172.21.15.37:0/2209506975 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff5565a9700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 0x7ff5501997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.491+0000 7ff555da8700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 0x7ff55019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 -- 172.21.15.37:0/2209506975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 msgr2=0x7ff5501997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:39.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 0x7ff5501997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 -- 172.21.15.37:0/2209506975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 msgr2=0x7ff55019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 0x7ff55019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 -- 172.21.15.37:0/2209506975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff550199fd0 con 0x7ff550102ce0 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5565a9700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 0x7ff5501997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff555da8700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 0x7ff55019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff5555a7700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c9b0 tx=0x7ff54c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c007b00 con 0x7ff550102ce0 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55019a2c0 con 0x7ff550102ce0 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c007ca0 con 0x7ff550102ce0 2024-04-26T18:30:39.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c018730 con 0x7ff550102ce0 2024-04-26T18:30:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.492+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5501a64d0 con 0x7ff550102ce0 2024-04-26T18:30:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.493+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff550103a40 con 0x7ff550102ce0 2024-04-26T18:30:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.494+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff54c00f450 con 0x7ff550102ce0 2024-04-26T18:30:39.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.495+0000 7ff542ffd700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 0x7ff53c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.495+0000 7ff555da8700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 0x7ff53c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.495+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7ff54c0d0c30 con 0x7ff550102ce0 2024-04-26T18:30:39.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.496+0000 7ff555da8700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 0x7ff53c0a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff544005fd0 tx=0x7ff544005db0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:39.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.499+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff54c014030 con 0x7ff550102ce0 2024-04-26T18:30:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:39 smithi120 ceph-mon[32244]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:39 smithi170 ceph-mon[32697]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:39.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.757+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff55006a110 con 0x7ff550102ce0 2024-04-26T18:30:39.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.758+0000 7ff542ffd700 1 -- 172.21.15.37:0/2209506975 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11884 (secure 0 0 0) 0x7ff54c09d2f0 con 0x7ff550102ce0 2024-04-26T18:30:39.758 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:30:39.758 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:30:27.309322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/2099915678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":2099915678},{"type":"v1","addr":"172.21.15.170:6837","nonce":2099915678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24449,24442]},"id":1}]} 2024-04-26T18:30:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 msgr2=0x7ff53c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 0x7ff53c0a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff544005fd0 tx=0x7ff544005db0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 msgr2=0x7ff55019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c9b0 tx=0x7ff54c00cd70 comp rx=0 tx=0).stop 2024-04-26T18:30:39.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 shutdown_connections 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff550108390 0x7ff5501997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff53c0a3db0 0x7ff53c0a6270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff550102320 0x7ff55019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 --2- 172.21.15.37:0/2209506975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff550102ce0 0x7ff55019fbb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 >> 172.21.15.37:0/2209506975 conn(0x7ff5500792d0 msgr2=0x7ff550101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 shutdown_connections 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:39.760+0000 7ff55800c700 1 -- 172.21.15.37:0/2209506975 wait complete. 2024-04-26T18:30:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-04-26T18:30:40.404 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:40 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2209506975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:40.436 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24464 quiesce db --cancel --all' 2024-04-26T18:30:40.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24464 quiesce db --cancel --all 2024-04-26T18:30:40.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:40 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2209506975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:40.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:40 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2209506975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:40.695 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:30:41.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:41 smithi170 ceph-mon[32697]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:41.477 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:41 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:41 smithi037 ceph-mon[29171]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:41 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:41 smithi120 ceph-mon[32244]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 289 op/s 2024-04-26T18:30:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:41 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:42.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:42 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 -- 172.21.15.37:0/1938549707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102ce0 msgr2=0x7f0cf810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/1938549707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102ce0 0x7f0cf810f1d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b0a0 tx=0x7f0cf000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 -- 172.21.15.37:0/1938549707 shutdown_connections 2024-04-26T18:30:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/1938549707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102ce0 0x7f0cf810f1d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/1938549707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/1938549707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102320 0x7f0cf81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 -- 172.21.15.37:0/1938549707 >> 172.21.15.37:0/1938549707 conn(0x7f0cf80792d0 msgr2=0x7f0cf80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 -- 172.21.15.37:0/1938549707 shutdown_connections 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.491+0000 7f0cfdd32700 1 -- 172.21.15.37:0/1938549707 wait complete. 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.492+0000 7f0cfdd32700 1 Processor -- start 2024-04-26T18:30:42.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.492+0000 7f0cfdd32700 1 -- start start 2024-04-26T18:30:42.493 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.7 2024-04-26T18:30:42.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:30:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.492+0000 7f0cfdd32700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.492+0000 7f0cfdd32700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102ce0 0x7f0cf8104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.492+0000 7f0cfdd32700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cfdd32700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81124b0 con 0x7f0cf8102320 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cfdd32700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112330 con 0x7f0cf8108390 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cfdd32700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112630 con 0x7f0cf8102ce0 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37646/0 (socket says 172.21.15.37:37646) 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 -- 172.21.15.37:0/3039098090 learned_addr learned my addr 172.21.15.37:0/3039098090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102ce0 0x7f0cf8104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 -- 172.21.15.37:0/3039098090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102ce0 msgr2=0x7f0cf8104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102ce0 0x7f0cf8104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 -- 172.21.15.37:0/3039098090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 msgr2=0x7f0cf8104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 -- 172.21.15.37:0/3039098090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a7f80 con 0x7f0cf8102320 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c920 tx=0x7f0ce800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf4ff9700 1 -- 172.21.15.37:0/3039098090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8007b00 con 0x7f0cf8102320 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a8270 con 0x7f0cf8102320 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf4ff9700 1 -- 172.21.15.37:0/3039098090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce8004ce0 con 0x7f0cf8102320 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cf4ff9700 1 -- 172.21.15.37:0/3039098090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8005650 con 0x7f0cf8102320 2024-04-26T18:30:42.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.493+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a87b0 con 0x7f0cf8102320 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.496+0000 7f0cf4ff9700 1 -- 172.21.15.37:0/3039098090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ce8007ca0 con 0x7f0cf8102320 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.496+0000 7f0cf4ff9700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 0x7f0ce00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 0x7f0ce00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf4ff9700 1 -- 172.21.15.37:0/3039098090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0ce80d0770 con 0x7f0cf8102320 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 Processor -- start 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 0x7f0ce00a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005fd0 tx=0x7f0cec005f60 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 -- start start 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558937c927b0 con 0x558937d4a8e0 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558937c31650 con 0x558937d3f110 2024-04-26T18:30:42.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558937c8e770 con 0x558937d3fa30 2024-04-26T18:30:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37656/0 (socket says 172.21.15.37:37656) 2024-04-26T18:30:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.497+0000 7f0cf7fff700 1 -- 172.21.15.37:0/2133088284 learned_addr learned my addr 172.21.15.37:0/2133088284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d3f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 -- 172.21.15.37:0/2133088284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 msgr2=0x558937d4a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 -- 172.21.15.37:0/2133088284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 msgr2=0x558937d3f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d3f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 -- 172.21.15.37:0/2133088284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558937bdf940 con 0x558937d4a8e0 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b830 tx=0x7f0cf00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff6515028d2d6ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cddffb700 1 -- 172.21.15.37:0/2133088284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0022070 con 0x558937d4a8e0 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cddffb700 1 -- 172.21.15.37:0/2133088284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf00074e0 con 0x558937d4a8e0 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.498+0000 7f0cddffb700 1 -- 172.21.15.37:0/2133088284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf001c640 con 0x558937d4a8e0 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- 172.21.15.37:0/2133088284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 msgr2=0x558937d4ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000b830 tx=0x7f0cf00092e0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- 172.21.15.37:0/2133088284 shutdown_connections 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d4a8e0 0x558937d4ccd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d3f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/2133088284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- 172.21.15.37:0/2133088284 >> 172.21.15.37:0/2133088284 conn(0x558937c6cdb0 msgr2=0x558937d3b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- 172.21.15.37:0/2133088284 shutdown_connections 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- 172.21.15.37:0/2133088284 wait complete. 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.499+0000 7f0d085c4b80 1 -- start start 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 Processor -- start 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d4a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558937d4d450 con 0x558937d3fa30 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558937bf89c0 con 0x558937d3f110 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558937d4d5d0 con 0x558937d4a8e0 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37658/0 (socket says 172.21.15.37:37658) 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf6ffd700 1 -- 172.21.15.37:0/3893459334 learned_addr learned my addr 172.21.15.37:0/3893459334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0d085c4b80 10 client.0 _ll_register_callbacks cb 0x558937c265f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d4a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf6ffd700 1 -- 172.21.15.37:0/3893459334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 msgr2=0x558937d47890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.500+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d47890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cf6ffd700 1 -- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 msgr2=0x558937d4a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d4a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cf6ffd700 1 -- 172.21.15.37:0/3893459334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558937d47f40 con 0x558937d3fa30 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b960 tx=0x7f0cec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aa37e5a7d039fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00fe70 con 0x558937d3fa30 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0cec01f430 con 0x558937d3fa30 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf00074e0 con 0x558937d3fa30 2024-04-26T18:30:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cec01f630 con 0x558937d3fa30 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0d085c4b80 10 client.16044 fetch_fsmap learned FSMap version 32 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558937d57f10 con 0x558937d3fa30 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.501+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf00074e0 con 0x558937d3fa30 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f0cec0308d0 con 0x558937d3fa30 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 10 client.16044 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 20 client.16044 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937d5a4a0 0x558937d5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 4 client.16044 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558937ca0190 con 0x558937d5a4a0 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.502+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937d5a4a0 0x558937d5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.503+0000 7f0cd6ffd700 10 client.16044 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.503+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937d5a4a0 0x558937d5c8b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab6a5ef2fb8f691 server_cookie=20b0f2bf7f5eceb0 in_seq=0 out_seq=0 2024-04-26T18:30:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.503+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/3893459334 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558937ca0190 con 0x558937d5a4a0 2024-04-26T18:30:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.503+0000 7f0cd6ffd700 10 client.16044 handle_command_reply: tid=0 2024-04-26T18:30:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.503+0000 7f0d085c4b80 1 client.16044 shutdown 2024-04-26T18:30:42.508 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:42 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 msgr2=0x558937d47350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b960 tx=0x7f0cec004750 comp rx=0 tx=0).stop 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 shutdown_connections 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937d5a4a0 0x558937d5c8b0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d47890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3f110 0x558937d4a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/3893459334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3fa30 0x558937d47350 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 >> 172.21.15.37:0/3893459334 conn(0x558937c6cdb0 msgr2=0x558937d3b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0cd6ffd700 0 client.16044 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:42.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 shutdown_connections 2024-04-26T18:30:42.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 1 -- 172.21.15.37:0/3893459334 wait complete. 2024-04-26T18:30:42.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.504+0000 7f0d085c4b80 20 client.16044 trim_cache size 0 max 16384 2024-04-26T18:30:42.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 Processor -- start 2024-04-26T18:30:42.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 -- start start 2024-04-26T18:30:42.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3fa30 0x558937d42900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d49b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b2840 con 0x558937d3f110 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b0070 con 0x558937d3fa30 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81adec0 con 0x558937d4a8e0 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37662/0 (socket says 172.21.15.37:37662) 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.525+0000 7f0cf77fe700 1 -- 172.21.15.37:0/1069249715 learned_addr learned my addr 172.21.15.37:0/1069249715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d49b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3fa30 0x558937d42900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 -- 172.21.15.37:0/1069249715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 msgr2=0x558937d49b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d49b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 -- 172.21.15.37:0/1069249715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3fa30 msgr2=0x558937d42900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3fa30 0x558937d42900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 -- 172.21.15.37:0/1069249715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558937d32eb0 con 0x558937d3f110 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80990d0 tx=0x7f0ce8087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a286b4d3ea4dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cdd7fa700 1 -- 172.21.15.37:0/1069249715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801d040 con 0x558937d3f110 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cdd7fa700 1 -- 172.21.15.37:0/1069249715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce80870b0 con 0x558937d3f110 2024-04-26T18:30:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.526+0000 7f0cdd7fa700 1 -- 172.21.15.37:0/1069249715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce809c960 con 0x558937d3f110 2024-04-26T18:30:42.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1069249715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 msgr2=0x558937d45cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80990d0 tx=0x7f0ce8087040 comp rx=0 tx=0).stop 2024-04-26T18:30:42.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1069249715 shutdown_connections 2024-04-26T18:30:42.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d4a8e0 0x558937d49b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d45cc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1069249715 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d3fa30 0x558937d42900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1069249715 >> 172.21.15.37:0/1069249715 conn(0x558937c6cdb0 msgr2=0x558937d421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1069249715 shutdown_connections 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.527+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1069249715 wait complete. 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 -- start start 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 Processor -- start 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 0x558937d4f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81ae530 con 0x558937d3f110 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81af9a0 con 0x558937d4a8e0 2024-04-26T18:30:42.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81aec00 con 0x558937d3fa30 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.528+0000 7f0d085c4b80 10 client.0 _ll_register_callbacks cb 0x558937c265f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 0x558937d4f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37672/0 (socket says 172.21.15.37:37672) 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf77fe700 1 -- 172.21.15.37:0/1929109925 learned_addr learned my addr 172.21.15.37:0/1929109925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf6ffd700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 -- 172.21.15.37:0/1929109925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 msgr2=0x558937d4ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 -- 172.21.15.37:0/1929109925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 msgr2=0x558937d52a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 -- 172.21.15.37:0/1929109925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558937d4fa10 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cf7fff700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 0x558937d4f360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006130 tx=0x7f0cf000b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d35a9ace758ffcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.529+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0022040 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce80870b0 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f0cf00076f0 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cf0007910 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 10 client.25658 fetch_fsmap learned FSMap version 32 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558937dc18c0 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce80870b0 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f0cf0011070 con 0x558937d4a8e0 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T18:30:42.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 10 client.25658 resolve_mds: validated gid 24464 aka daemon mds.b 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 20 client.25658 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937dc1b40 0x558937dc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 4 client.25658 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.530+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x558937c9a230 con 0x558937dc1b40 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937dc1b40 0x558937dc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0cf77fe700 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937dc1b40 0x558937dc3f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca92ddc2b5f20200 server_cookie=69663a5a476aa05f in_seq=0 out_seq=0 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0cd6ffd700 10 client.25658 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0cd6ffd700 1 -- 172.21.15.37:0/1929109925 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x558937c9a230 con 0x558937dc1b40 2024-04-26T18:30:42.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0cd6ffd700 10 client.25658 handle_command_reply: tid=0 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0d085c4b80 1 client.25658 shutdown 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 msgr2=0x558937d4f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.531+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 0x558937d4f360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006130 tx=0x7f0cf000b460 comp rx=0 tx=0).stop 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 shutdown_connections 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x558937dc1b40 0x558937dc3f30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558937d4a8e0 0x558937d4f360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558937d3f110 0x558937d52a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 --2- 172.21.15.37:0/1929109925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558937d3fa30 0x558937d4ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 >> 172.21.15.37:0/1929109925 conn(0x558937c6cdb0 msgr2=0x558937d55bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0cd6ffd700 0 client.25658 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 shutdown_connections 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 1 -- 172.21.15.37:0/1929109925 wait complete. 2024-04-26T18:30:42.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0d085c4b80 20 client.25658 trim_cache size 0 max 16384 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 32, 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24464, 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-04-26T18:30:42.538 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 msgr2=0x7f0ce00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 0x7f0ce00a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005fd0 tx=0x7f0cec005f60 comp rx=0 tx=0).stop 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 msgr2=0x7f0cf8104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.532+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c920 tx=0x7f0ce800cce0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 shutdown_connections 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f0ce00a3db0 0x7f0ce00a6270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cf8108390 0x7f0cf8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf8102320 0x7f0cf8104000 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 --2- 172.21.15.37:0/3039098090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf8102ce0 0x7f0cf8104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 >> 172.21.15.37:0/3039098090 conn(0x7f0cf80792d0 msgr2=0x7f0cf8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 shutdown_connections 2024-04-26T18:30:42.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:42.533+0000 7f0cfdd32700 1 -- 172.21.15.37:0/3039098090 wait complete. 2024-04-26T18:30:42.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:42 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:43.209 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:30:43.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:43 smithi037 ceph-mon[29171]: pgmap v194: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 394 op/s 2024-04-26T18:30:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:43 smithi120 ceph-mon[32244]: pgmap v194: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 394 op/s 2024-04-26T18:30:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:43 smithi170 ceph-mon[32697]: pgmap v194: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 394 op/s 2024-04-26T18:30:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- 172.21.15.37:0/572703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 msgr2=0x7f3ac4101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/572703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4101cd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009930 tx=0x7f3ab402a420 comp rx=0 tx=0).stop 2024-04-26T18:30:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- 172.21.15.37:0/572703113 shutdown_connections 2024-04-26T18:30:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/572703113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/572703113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4101cd0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/572703113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac4107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- 172.21.15.37:0/572703113 >> 172.21.15.37:0/572703113 conn(0x7f3ac4077bb0 msgr2=0x7f3ac4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- 172.21.15.37:0/572703113 shutdown_connections 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- 172.21.15.37:0/572703113 wait complete. 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 Processor -- start 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.943+0000 7f3acb1e6640 1 -- start start 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4111b20 con 0x7f3ac4101850 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41119a0 con 0x7f3ac4107860 2024-04-26T18:30:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4111ca0 con 0x7f3ac4102210 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37690/0 (socket says 172.21.15.37:37690) 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 -- 172.21.15.37:0/1903494492 learned_addr learned my addr 172.21.15.37:0/1903494492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3abbfff640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 -- 172.21.15.37:0/1903494492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 msgr2=0x7f3ac4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 -- 172.21.15.37:0/1903494492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 msgr2=0x7f3ac41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 -- 172.21.15.37:0/1903494492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac41a7500 con 0x7f3ac4101850 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3abbfff640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000d950 tx=0x7f3ac000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ab9ffb640 1 -- 172.21.15.37:0/1903494492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0017030 con 0x7f3ac4101850 2024-04-26T18:30:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ab9ffb640 1 -- 172.21.15.37:0/1903494492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac0004020 con 0x7f3ac4101850 2024-04-26T18:30:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac41a77f0 con 0x7f3ac4101850 2024-04-26T18:30:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.944+0000 7f3ab9ffb640 1 -- 172.21.15.37:0/1903494492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00047a0 con 0x7f3ac4101850 2024-04-26T18:30:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.945+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41a7d60 con 0x7f3ac4101850 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.947+0000 7f3ab9ffb640 1 -- 172.21.15.37:0/1903494492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ac0009a30 con 0x7f3ac4101850 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 Processor -- start 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 -- start start 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3ab9ffb640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 0x7f3aac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 0x7f3aac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae46c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 0x555bae4794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555bae3d2460 con 0x555bae46c820 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555bae38c590 con 0x555bae4770e0 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3ab9ffb640 1 -- 172.21.15.37:0/1903494492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3ac00d0280 con 0x7f3ac4101850 2024-04-26T18:30:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bae3ce420 con 0x555bae46bf00 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3abbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3abbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37702/0 (socket says 172.21.15.37:37702) 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3abbfff640 1 -- 172.21.15.37:0/1383104149 learned_addr learned my addr 172.21.15.37:0/1383104149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.948+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 0x7f3aac0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005fd0 tx=0x7f3abc010740 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae46c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 0x555bae4794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 -- 172.21.15.37:0/1383104149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 msgr2=0x555bae46c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae46c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 -- 172.21.15.37:0/1383104149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 msgr2=0x555bae4794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 0x555bae4794d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 -- 172.21.15.37:0/1383104149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bae3990d0 con 0x555bae46c820 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae46c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:43.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3ac9783640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 0x555bae4794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3abbfff640 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3ab402a940 tx=0x7f3ab4032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405c048830ec2fed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3aaaffd640 1 -- 172.21.15.37:0/1383104149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab403f070 con 0x555bae46c820 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3aaaffd640 1 -- 172.21.15.37:0/1383104149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab40316a0 con 0x555bae46c820 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.949+0000 7f3aaaffd640 1 -- 172.21.15.37:0/1383104149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4035070 con 0x555bae46c820 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- 172.21.15.37:0/1383104149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 msgr2=0x555bae476ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3ab402a940 tx=0x7f3ab4032cd0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- 172.21.15.37:0/1383104149 shutdown_connections 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae4770e0 0x555bae4794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46c820 0x555bae476ba0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/1383104149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae46c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- 172.21.15.37:0/1383104149 >> 172.21.15.37:0/1383104149 conn(0x555bae468410 msgr2=0x555bae4687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- 172.21.15.37:0/1383104149 shutdown_connections 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- 172.21.15.37:0/1383104149 wait complete. 2024-04-26T18:30:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- start start 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 Processor -- start 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae474080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae4745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.950+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555bae479cb0 con 0x555bae4770e0 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555bae359b60 con 0x555bae46c820 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bae479e30 con 0x555bae46bf00 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3acdfd8740 10 client.0 _ll_register_callbacks cb 0x555bae37e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37708/0 (socket says 172.21.15.37:37708) 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 -- 172.21.15.37:0/3866227365 learned_addr learned my addr 172.21.15.37:0/3866227365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3abbfff640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae4745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae474080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 -- 172.21.15.37:0/3866227365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 msgr2=0x555bae474080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae474080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 -- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 msgr2=0x555bae4745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae4745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 -- 172.21.15.37:0/3866227365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bae0e4210 con 0x555bae4770e0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae474080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3abc013990 tx=0x7f3abc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e234e8bb7666b138 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3aa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc011960 con 0x555bae4770e0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3abc011b00 con 0x555bae4770e0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.951+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab40316a0 con 0x555bae4770e0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3abc011d00 con 0x555bae4770e0 2024-04-26T18:30:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 10 client.16062 fetch_fsmap learned FSMap version 32 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555bae1b0620 con 0x555bae4770e0 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab40316a0 con 0x555bae4770e0 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f3abc00f040 con 0x555bae4770e0 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 10 client.16062 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 20 client.16062 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae487200 0x555bae489650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 4 client.16062 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bae3e0070 con 0x555bae487200 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.952+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae487200 0x555bae489650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.953+0000 7f3aa3fff640 10 client.16062 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.953+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae487200 0x555bae489650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89dcbb3fcdcb82e server_cookie=3ea108a881fb7dbf in_seq=0 out_seq=0 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3aa3fff640 1 -- 172.21.15.37:0/3866227365 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bae3e0070 con 0x555bae487200 2024-04-26T18:30:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3aa3fff640 10 client.16062 handle_command_reply: tid=0 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 client.16062 shutdown 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 msgr2=0x555bae474b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3abc013990 tx=0x7f3abc004ab0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 shutdown_connections 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae487200 0x555bae489650 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae4745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae474b00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3866227365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46bf00 0x555bae474080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3aa3fff640 0 client.16062 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 >> 172.21.15.37:0/3866227365 conn(0x555bae468410 msgr2=0x555bae46e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 shutdown_connections 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3866227365 wait complete. 2024-04-26T18:30:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.954+0000 7f3acdfd8740 20 client.16062 trim_cache size 0 max 16384 2024-04-26T18:30:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 Processor -- start 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 -- start start 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46bf00 0x555bae482b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46c820 0x555bae46e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b2fe0 con 0x555bae4770e0 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b0850 con 0x555bae46bf00 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.969+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41ae6a0 con 0x555bae46c820 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37716/0 (socket says 172.21.15.37:37716) 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 -- 172.21.15.37:0/3838966414 learned_addr learned my addr 172.21.15.37:0/3838966414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46bf00 0x555bae482b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3abbfff640 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46c820 0x555bae46e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 -- 172.21.15.37:0/3838966414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46c820 msgr2=0x555bae46e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46c820 0x555bae46e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 -- 172.21.15.37:0/3838966414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46bf00 msgr2=0x555bae482b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46bf00 0x555bae482b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 -- 172.21.15.37:0/3838966414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bae473070 con 0x555bae4770e0 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3ac9783640 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3abc013960 tx=0x7f3abc013640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9473692ffa2001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/3838966414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc03f070 con 0x555bae4770e0 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/3838966414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc00f040 con 0x555bae4770e0 2024-04-26T18:30:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.970+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/3838966414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc034040 con 0x555bae4770e0 2024-04-26T18:30:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3838966414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 msgr2=0x555bae476140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3abc013960 tx=0x7f3abc013640 comp rx=0 tx=0).stop 2024-04-26T18:30:43.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3838966414 shutdown_connections 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae4770e0 0x555bae476140 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae46c820 0x555bae46e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/3838966414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46bf00 0x555bae482b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3838966414 >> 172.21.15.37:0/3838966414 conn(0x555bae468410 msgr2=0x555bae4687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3838966414 shutdown_connections 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- 172.21.15.37:0/3838966414 wait complete. 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 -- start start 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.971+0000 7f3acdfd8740 1 Processor -- start 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae471700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 0x555bae4ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41aed10 con 0x555bae46bf00 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41b0180 con 0x555bae46c820 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac41af3e0 con 0x555bae4770e0 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 10 client.0 _ll_register_callbacks cb 0x7f3abc010e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37726/0 (socket says 172.21.15.37:37726) 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 -- 172.21.15.37:0/266202138 learned_addr learned my addr 172.21.15.37:0/266202138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac9783640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 0x555bae4ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3abbfff640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae471700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 -- 172.21.15.37:0/266202138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 msgr2=0x555bae4ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 0x555bae4ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 -- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 msgr2=0x555bae471700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae471700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 -- 172.21.15.37:0/266202138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bae4ecd10 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac9783640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 0x555bae4ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3ac8f82640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0098b80 tx=0x7f3ac009bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd63e5255fc6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3aa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00ba060 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f3ac00041d0 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.972+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3abc00f040 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ac00a4e20 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 10 client.16068 fetch_fsmap learned FSMap version 32 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555bae4edf40 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3abc00f040 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f3ac00cdc00 con 0x555bae46bf00 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 10 client.16068 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 20 client.16068 populate_metadata read hostname 'smithi037' 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae4ee650 0x555bae4f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 4 client.16068 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 --> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bae3d9ee0 con 0x555bae4ee650 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.973+0000 7f3abbfff640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae4ee650 0x555bae4f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.974+0000 7f3aa3fff640 10 client.16068 ms_handle_connect on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:43.974+0000 7f3abbfff640 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae4ee650 0x555bae4f0a70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a514a31ac7706bae server_cookie=78cf5ef6898b5aeb in_seq=0 out_seq=0 2024-04-26T18:30:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3aa3fff640 1 -- 172.21.15.37:0/266202138 <== mds.0 v2:172.21.15.120:6838/1246417570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bae3d9ee0 con 0x555bae4ee650 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3aa3fff640 10 client.16068 handle_command_reply: tid=0 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 client.16068 shutdown 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 msgr2=0x555bae4711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0098b80 tx=0x7f3ac009bf80 comp rx=0 tx=0).stop 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 shutdown_connections 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:6838/1246417570,v1:172.21.15.120:6839/1246417570] conn(0x555bae4ee650 0x555bae4f0a70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555bae46c820 0x555bae471700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bae4770e0 0x555bae4ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 --2- 172.21.15.37:0/266202138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555bae46bf00 0x555bae4711c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3aa3fff640 0 client.16068 ms_handle_reset on v2:172.21.15.120:6838/1246417570 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 >> 172.21.15.37:0/266202138 conn(0x555bae468410 msgr2=0x555bae4675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 shutdown_connections 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 1 -- 172.21.15.37:0/266202138 wait complete. 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.000+0000 7f3acdfd8740 20 client.16068 trim_cache size 0 max 16384 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T18:30:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 msgr2=0x7f3aac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 0x7f3aac0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3abc005fd0 tx=0x7f3abc010740 comp rx=0 tx=0).stop 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 msgr2=0x7f3ac4197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000d950 tx=0x7f3ac000de10 comp rx=0 tx=0).stop 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 shutdown_connections 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f3aac0a3e40 0x7f3aac0a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ac4107860 0x7f3ac41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ac4102210 0x7f3ac4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 --2- 172.21.15.37:0/1903494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac4101850 0x7f3ac4197c50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 >> 172.21.15.37:0/1903494492 conn(0x7f3ac4077bb0 msgr2=0x7f3ac40ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 shutdown_connections 2024-04-26T18:30:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T18:30:44.001+0000 7f3acb1e6640 1 -- 172.21.15.37:0/1903494492 wait complete. 2024-04-26T18:30:44.825 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:30:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: ignoring --setuser ceph since I am not root 2024-04-26T18:30:44.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:30:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: ignoring --setgroup ceph since I am not root 2024-04-26T18:30:44.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:30:44 smithi170 ceph-mds[63003]: -- 172.21.15.170:0/1258847519 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7b0d2ba40 con 0x55b7b0cfe800 2024-04-26T18:30:44.826 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:30:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: starting mds.c at 2024-04-26T18:30:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.027+0000 7f8365b0a700 1 -- 172.21.15.37:0/1376910069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83601025a0 msgr2=0x7f8360102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.027+0000 7f8365b0a700 1 --2- 172.21.15.37:0/1376910069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83601025a0 0x7f8360102980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f83500099c0 tx=0x7f8350009cd0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 -- 172.21.15.37:0/1376910069 shutdown_connections 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 --2- 172.21.15.37:0/1376910069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8360107680 0x7f836010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 --2- 172.21.15.37:0/1376910069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83601025a0 0x7f8360102980 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 --2- 172.21.15.37:0/1376910069 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360102f50 0x7f8360106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 -- 172.21.15.37:0/1376910069 >> 172.21.15.37:0/1376910069 conn(0x7f83600793c0 msgr2=0x7f83600797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 -- 172.21.15.37:0/1376910069 shutdown_connections 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 -- 172.21.15.37:0/1376910069 wait complete. 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.028+0000 7f8365b0a700 1 Processor -- start 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 -- start start 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83601025a0 0x7f83601a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:45.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 0x7f836019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8360112540 con 0x7f8360102f50 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83601123c0 con 0x7f8360107680 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f8365b0a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83601126c0 con 0x7f83601025a0 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40858/0 (socket says 172.21.15.37:40858) 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 -- 172.21.15.37:0/4265440684 learned_addr learned my addr 172.21.15.37:0/4265440684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835f7fe700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83601025a0 0x7f83601a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835ffff700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 0x7f836019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 -- 172.21.15.37:0/4265440684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83601025a0 msgr2=0x7f83601a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83601025a0 0x7f83601a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 -- 172.21.15.37:0/4265440684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 msgr2=0x7f836019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:45.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 0x7f836019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 -- 172.21.15.37:0/4265440684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019c410 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835ffff700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 0x7f836019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.029+0000 7f835effd700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f835400d980 tx=0x7f835400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354017070 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019c700 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8354009de0 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835400f3c0 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a89e0 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.030+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8360103a80 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.032+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8354009930 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.032+0000 7f835cff9700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 0x7f83480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.033+0000 7f835f7fe700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 0x7f83480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.033+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8535+0+0 (secure 0 0 0) 0x7f83540d0660 con 0x7f8360102f50 2024-04-26T18:30:45.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.033+0000 7f835f7fe700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 0x7f83480a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8350009990 tx=0x7f835000b790 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:45.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.036+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f835409ccf0 con 0x7f8360102f50 2024-04-26T18:30:45.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.301+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f836019cff0 con 0x7f8360102f50 2024-04-26T18:30:45.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.302+0000 7f835cff9700 1 -- 172.21.15.37:0/4265440684 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11886 (secure 0 0 0) 0x7f835409c5f0 con 0x7f8360102f50 2024-04-26T18:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:30:45.074860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25618":{"gid":25618,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/1357982747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1357982747},{"type":"v1","addr":"172.21.15.170:6837","nonce":1357982747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 msgr2=0x7f83480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 0x7f83480a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8350009990 tx=0x7f835000b790 comp rx=0 tx=0).stop 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 msgr2=0x7f83601a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f835400d980 tx=0x7f835400dc90 comp rx=0 tx=0).stop 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 shutdown_connections 2024-04-26T18:30:45.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f83480a3db0 0x7f83480a6270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8360107680 0x7f836019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83601025a0 0x7f83601a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 --2- 172.21.15.37:0/4265440684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8360102f50 0x7f83601a1fa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 >> 172.21.15.37:0/4265440684 conn(0x7f83600793c0 msgr2=0x7f83601015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 shutdown_connections 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:30:45.304+0000 7f8365b0a700 1 -- 172.21.15.37:0/4265440684 wait complete. 2024-04-26T18:30:45.305 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: pgmap v195: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: daemon mds.c restarted 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6836/1357982747,v1:172.21.15.170:6837/1357982747] up:boot 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:45 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:30:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: pgmap v195: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-04-26T18:30:45.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: daemon mds.c restarted 2024-04-26T18:30:45.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:30:45.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6836/1357982747,v1:172.21.15.170:6837/1357982747] up:boot 2024-04-26T18:30:45.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:45.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:45 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: pgmap v195: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: daemon mds.c restarted 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6836/1357982747,v1:172.21.15.170:6837/1357982747] up:boot 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:30:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:45 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:30:45.972 INFO:tasks.quiescer:done joining Quiescers 2024-04-26T18:30:45.972 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T18:30:45.983 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:30:46.022 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T18:30:46.033 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T18:30:46.033 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:30:46.033 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:30:46.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T18:30:46.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T18:30:46.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:46 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4265440684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:46.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:46 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4265440684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:46.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:46 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4265440684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:47.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:47 smithi120 ceph-mon[32244]: pgmap v197: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:47 smithi170 ceph-mon[32697]: pgmap v197: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:47.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:47 smithi037 ceph-mon[29171]: pgmap v197: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:49.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:49 smithi120 ceph-mon[32244]: pgmap v198: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:49.535 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:49 smithi037 ceph-mon[29171]: pgmap v198: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:49 smithi170 ceph-mon[32697]: pgmap v198: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:51.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:51 smithi120 ceph-mon[32244]: pgmap v199: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:51 smithi170 ceph-mon[32697]: pgmap v199: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:51.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:51 smithi037 ceph-mon[29171]: pgmap v199: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 385 KiB/s wr, 140 op/s 2024-04-26T18:30:53.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:53 smithi120 ceph-mon[32244]: pgmap v200: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:53 smithi170 ceph-mon[32697]: pgmap v200: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:53.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:53 smithi037 ceph-mon[29171]: pgmap v200: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:55.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:55 smithi120 ceph-mon[32244]: pgmap v201: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:55 smithi170 ceph-mon[32697]: pgmap v201: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:55.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:55 smithi037 ceph-mon[29171]: pgmap v201: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:30:57.505 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:57 smithi120 ceph-mon[32244]: pgmap v202: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-04-26T18:30:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:57 smithi170 ceph-mon[32697]: pgmap v202: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-04-26T18:30:57.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:57 smithi037 ceph-mon[29171]: pgmap v202: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-04-26T18:30:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:30:59 smithi170 ceph-mon[32697]: pgmap v203: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:30:59.590 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:30:59 smithi037 ceph-mon[29171]: pgmap v203: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:30:59.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:30:59 smithi120 ceph-mon[32244]: pgmap v203: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:31:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:01 smithi170 ceph-mon[32697]: pgmap v204: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:31:01.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:01 smithi037 ceph-mon[29171]: pgmap v204: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:31:01.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:01 smithi120 ceph-mon[32244]: pgmap v204: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T18:31:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:03 smithi170 ceph-mon[32697]: pgmap v205: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T18:31:03.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:03 smithi037 ceph-mon[29171]: pgmap v205: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T18:31:03.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:03 smithi120 ceph-mon[32244]: pgmap v205: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-26T18:31:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:05 smithi170 ceph-mon[32697]: pgmap v206: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s 2024-04-26T18:31:05.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:05 smithi037 ceph-mon[29171]: pgmap v206: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s 2024-04-26T18:31:05.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:05 smithi120 ceph-mon[32244]: pgmap v206: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4 op/s 2024-04-26T18:31:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:07 smithi170 ceph-mon[32697]: pgmap v207: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:31:07.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:07 smithi037 ceph-mon[29171]: pgmap v207: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:31:07.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:07 smithi120 ceph-mon[32244]: pgmap v207: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:31:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:09 smithi170 ceph-mon[32697]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:09.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:09 smithi037 ceph-mon[29171]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:09.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:09 smithi120 ceph-mon[32244]: pgmap v208: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:11 smithi170 ceph-mon[32697]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:11.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:11 smithi037 ceph-mon[29171]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:11.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:11 smithi120 ceph-mon[32244]: pgmap v209: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:31:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:12 smithi170 ceph-mon[32697]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:31:12.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:12 smithi037 ceph-mon[29171]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:31:12.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:12 smithi120 ceph-mon[32244]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:31:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:13 smithi170 ceph-mon[32697]: pgmap v211: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:13.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:13 smithi037 ceph-mon[29171]: pgmap v211: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:13.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:13 smithi120 ceph-mon[32244]: pgmap v211: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:15.555 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:15 smithi120 ceph-mon[32244]: pgmap v212: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:15 smithi170 ceph-mon[32697]: pgmap v212: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:15 smithi037 ceph-mon[29171]: pgmap v212: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:31:16.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:16 smithi037 ceph-mon[29171]: pgmap v213: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:16.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:16 smithi120 ceph-mon[32244]: pgmap v213: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:16.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:16 smithi170 ceph-mon[32697]: pgmap v213: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:18.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:18 smithi037 ceph-mon[29171]: pgmap v214: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:19.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:18 smithi120 ceph-mon[32244]: pgmap v214: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:19.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:18 smithi170 ceph-mon[32697]: pgmap v214: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:21 smithi037 ceph-mon[29171]: pgmap v215: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:21 smithi037 ceph-mon[29171]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:31:21.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:21 smithi120 ceph-mon[32244]: pgmap v215: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:21.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:21 smithi120 ceph-mon[32244]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:31:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:21 smithi170 ceph-mon[32697]: pgmap v215: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:31:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:21 smithi170 ceph-mon[32697]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:31:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:23 smithi037 ceph-mon[29171]: pgmap v217: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:23.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:23 smithi120 ceph-mon[32244]: pgmap v217: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:23 smithi170 ceph-mon[32697]: pgmap v217: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:25 smithi037 ceph-mon[29171]: pgmap v218: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:25 smithi120 ceph-mon[32244]: pgmap v218: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:25 smithi170 ceph-mon[32697]: pgmap v218: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:31:27.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:27 smithi037 ceph-mon[29171]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:27 smithi120 ceph-mon[32244]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:27 smithi170 ceph-mon[32697]: pgmap v219: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:29 smithi037 ceph-mon[29171]: pgmap v220: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:29.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:29 smithi120 ceph-mon[32244]: pgmap v220: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:29 smithi170 ceph-mon[32697]: pgmap v220: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:31 smithi037 ceph-mon[29171]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:31 smithi120 ceph-mon[32244]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:31 smithi170 ceph-mon[32697]: pgmap v221: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:31:32.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:32 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:32.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:32 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:32.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:32 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:33 smithi037 ceph-mon[29171]: pgmap v222: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T18:31:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:33 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:33 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:33 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:31:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:33 smithi120 ceph-mon[32244]: pgmap v222: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T18:31:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:33 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:33 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:33 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:31:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:33 smithi170 ceph-mon[32697]: pgmap v222: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-26T18:31:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:33 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:33 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:33 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:31:35.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:35 smithi037 ceph-mon[29171]: pgmap v223: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:35 smithi120 ceph-mon[32244]: pgmap v223: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:35 smithi170 ceph-mon[32697]: pgmap v223: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:37.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:37 smithi037 ceph-mon[29171]: pgmap v224: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:37.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:37 smithi120 ceph-mon[32244]: pgmap v224: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:37 smithi170 ceph-mon[32697]: pgmap v224: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:39.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:39 smithi037 ceph-mon[29171]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:39 smithi120 ceph-mon[32244]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:39 smithi170 ceph-mon[32697]: pgmap v225: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:41.378 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:41 smithi170 ceph-mon[32697]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:41.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:41 smithi037 ceph-mon[29171]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:41.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:41 smithi120 ceph-mon[32244]: pgmap v226: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:43.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:43 smithi037 ceph-mon[29171]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:43.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:43 smithi120 ceph-mon[32244]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:43 smithi170 ceph-mon[32697]: pgmap v227: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:45 smithi037 ceph-mon[29171]: pgmap v228: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:45.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:45 smithi120 ceph-mon[32244]: pgmap v228: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:45 smithi170 ceph-mon[32697]: pgmap v228: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:47 smithi037 ceph-mon[29171]: pgmap v229: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:47 smithi120 ceph-mon[32244]: pgmap v229: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:47 smithi170 ceph-mon[32697]: pgmap v229: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:49.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:49 smithi037 ceph-mon[29171]: pgmap v230: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:49.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:49 smithi120 ceph-mon[32244]: pgmap v230: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:49 smithi170 ceph-mon[32697]: pgmap v230: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:51.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:51 smithi037 ceph-mon[29171]: pgmap v231: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:51 smithi120 ceph-mon[32244]: pgmap v231: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:51 smithi170 ceph-mon[32697]: pgmap v231: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:53.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:53 smithi037 ceph-mon[29171]: pgmap v232: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:53 smithi120 ceph-mon[32244]: pgmap v232: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:53 smithi170 ceph-mon[32697]: pgmap v232: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:55 smithi037 ceph-mon[29171]: pgmap v233: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:55 smithi120 ceph-mon[32244]: pgmap v233: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:55 smithi170 ceph-mon[32697]: pgmap v233: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:57.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:57 smithi037 ceph-mon[29171]: pgmap v234: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:57 smithi120 ceph-mon[32244]: pgmap v234: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:57 smithi170 ceph-mon[32697]: pgmap v234: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:31:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:31:59 smithi037 ceph-mon[29171]: pgmap v235: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:59.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:31:59 smithi120 ceph-mon[32244]: pgmap v235: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:31:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:31:59 smithi170 ceph-mon[32697]: pgmap v235: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:01.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:01 smithi037 ceph-mon[29171]: pgmap v236: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:01 smithi120 ceph-mon[32244]: pgmap v236: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:01 smithi170 ceph-mon[32697]: pgmap v236: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:03.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:03 smithi037 ceph-mon[29171]: pgmap v237: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:03 smithi120 ceph-mon[32244]: pgmap v237: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:03 smithi170 ceph-mon[32697]: pgmap v237: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:05.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:05 smithi037 ceph-mon[29171]: pgmap v238: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:05 smithi120 ceph-mon[32244]: pgmap v238: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:05 smithi170 ceph-mon[32697]: pgmap v238: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:07 smithi037 ceph-mon[29171]: pgmap v239: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:32:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:07 smithi120 ceph-mon[32244]: pgmap v239: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:32:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:07 smithi170 ceph-mon[32697]: pgmap v239: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:32:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:09 smithi037 ceph-mon[29171]: pgmap v240: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:09 smithi120 ceph-mon[32244]: pgmap v240: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:09 smithi170 ceph-mon[32697]: pgmap v240: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:11 smithi037 ceph-mon[29171]: pgmap v241: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:11 smithi120 ceph-mon[32244]: pgmap v241: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:11 smithi170 ceph-mon[32697]: pgmap v241: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:32:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:12 smithi037 ceph-mon[29171]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:32:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:12 smithi120 ceph-mon[32244]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:32:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:12 smithi170 ceph-mon[32697]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:32:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:13 smithi037 ceph-mon[29171]: pgmap v243: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:13.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:13 smithi120 ceph-mon[32244]: pgmap v243: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:13 smithi170 ceph-mon[32697]: pgmap v243: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:15 smithi037 ceph-mon[29171]: pgmap v244: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:15 smithi120 ceph-mon[32244]: pgmap v244: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:15 smithi170 ceph-mon[32697]: pgmap v244: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:32:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:17 smithi037 ceph-mon[29171]: pgmap v245: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:17 smithi120 ceph-mon[32244]: pgmap v245: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:17 smithi170 ceph-mon[32697]: pgmap v245: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:19.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:19 smithi037 ceph-mon[29171]: pgmap v246: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:19 smithi120 ceph-mon[32244]: pgmap v246: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:19 smithi170 ceph-mon[32697]: pgmap v246: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:21 smithi037 ceph-mon[29171]: pgmap v247: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:21 smithi037 ceph-mon[29171]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:32:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:21 smithi120 ceph-mon[32244]: pgmap v247: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:21.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:21 smithi120 ceph-mon[32244]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:32:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:21 smithi170 ceph-mon[32697]: pgmap v247: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:32:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:21 smithi170 ceph-mon[32697]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:32:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:23 smithi037 ceph-mon[29171]: pgmap v249: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:23 smithi120 ceph-mon[32244]: pgmap v249: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:23 smithi170 ceph-mon[32697]: pgmap v249: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:25 smithi037 ceph-mon[29171]: pgmap v250: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:25.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:25 smithi120 ceph-mon[32244]: pgmap v250: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:25 smithi170 ceph-mon[32697]: pgmap v250: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:32:27.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:27 smithi037 ceph-mon[29171]: pgmap v251: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:27 smithi120 ceph-mon[32244]: pgmap v251: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:27 smithi170 ceph-mon[32697]: pgmap v251: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:29 smithi037 ceph-mon[29171]: pgmap v252: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:29.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:29 smithi120 ceph-mon[32244]: pgmap v252: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:29 smithi170 ceph-mon[32697]: pgmap v252: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:31.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:31 smithi037 ceph-mon[29171]: pgmap v253: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:31.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:31 smithi120 ceph-mon[32244]: pgmap v253: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:31 smithi170 ceph-mon[32697]: pgmap v253: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:32:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:33 smithi037 ceph-mon[29171]: pgmap v254: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:32:33.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:33 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:33 smithi120 ceph-mon[32244]: pgmap v254: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:32:33.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:33 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:33 smithi170 ceph-mon[32697]: pgmap v254: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:32:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:33 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:32:34.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:34.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:34.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:32:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:34.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:34.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:32:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:35 smithi170 ceph-mon[32697]: pgmap v255: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:35.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:35 smithi037 ceph-mon[29171]: pgmap v255: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:35.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:35 smithi120 ceph-mon[32244]: pgmap v255: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:37.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:37 smithi170 ceph-mon[32697]: pgmap v256: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:37.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:37 smithi037 ceph-mon[29171]: pgmap v256: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:37.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:37 smithi120 ceph-mon[32244]: pgmap v256: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:39 smithi170 ceph-mon[32697]: pgmap v257: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:39.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:39 smithi037 ceph-mon[29171]: pgmap v257: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:39.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:39 smithi120 ceph-mon[32244]: pgmap v257: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:41 smithi170 ceph-mon[32697]: pgmap v258: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:41.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:41 smithi037 ceph-mon[29171]: pgmap v258: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:41.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:41 smithi120 ceph-mon[32244]: pgmap v258: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:32:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:43 smithi170 ceph-mon[32697]: pgmap v259: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:43.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:43 smithi037 ceph-mon[29171]: pgmap v259: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:43.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:43 smithi120 ceph-mon[32244]: pgmap v259: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:32:45.555 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:45 smithi120 ceph-mon[32244]: pgmap v260: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:45 smithi170 ceph-mon[32697]: pgmap v260: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:45.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:45 smithi037 ceph-mon[29171]: pgmap v260: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:47 smithi170 ceph-mon[32697]: pgmap v261: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:47.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:47 smithi037 ceph-mon[29171]: pgmap v261: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:47.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:47 smithi120 ceph-mon[32244]: pgmap v261: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:48.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:48 smithi037 ceph-mon[29171]: pgmap v262: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:48.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:48 smithi120 ceph-mon[32244]: pgmap v262: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:48.825 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:48 smithi170 ceph-mon[32697]: pgmap v262: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:50.982 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:50 smithi037 ceph-mon[29171]: pgmap v263: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:51.002 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:50 smithi120 ceph-mon[32244]: pgmap v263: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:51.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:50 smithi170 ceph-mon[32697]: pgmap v263: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:52.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:52 smithi037 ceph-mon[29171]: pgmap v264: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:53.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:52 smithi120 ceph-mon[32244]: pgmap v264: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:53.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:52 smithi170 ceph-mon[32697]: pgmap v264: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:54.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:54 smithi037 ceph-mon[29171]: pgmap v265: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:55.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:54 smithi120 ceph-mon[32244]: pgmap v265: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:55.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:54 smithi170 ceph-mon[32697]: pgmap v265: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:57.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:57 smithi037 ceph-mon[29171]: pgmap v266: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:57 smithi120 ceph-mon[32244]: pgmap v266: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:57 smithi170 ceph-mon[32697]: pgmap v266: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:32:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:32:59 smithi037 ceph-mon[29171]: pgmap v267: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:32:59 smithi120 ceph-mon[32244]: pgmap v267: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:32:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:32:59 smithi170 ceph-mon[32697]: pgmap v267: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:01.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:01 smithi037 ceph-mon[29171]: pgmap v268: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:01.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:01 smithi120 ceph-mon[32244]: pgmap v268: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:01 smithi170 ceph-mon[32697]: pgmap v268: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:03 smithi037 ceph-mon[29171]: pgmap v269: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:03 smithi120 ceph-mon[32244]: pgmap v269: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:03 smithi170 ceph-mon[32697]: pgmap v269: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:05.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:05 smithi037 ceph-mon[29171]: pgmap v270: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:33:05.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:05 smithi120 ceph-mon[32244]: pgmap v270: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:33:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:05 smithi170 ceph-mon[32697]: pgmap v270: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-26T18:33:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:07 smithi037 ceph-mon[29171]: pgmap v271: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:07 smithi037 ceph-mon[29171]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:33:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:07 smithi120 ceph-mon[32244]: pgmap v271: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:07 smithi120 ceph-mon[32244]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:33:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:07 smithi170 ceph-mon[32697]: pgmap v271: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:33:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:07 smithi170 ceph-mon[32697]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:33:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:09 smithi037 ceph-mon[29171]: pgmap v273: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:09 smithi120 ceph-mon[32244]: pgmap v273: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:09 smithi170 ceph-mon[32697]: pgmap v273: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:11.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:11 smithi037 ceph-mon[29171]: pgmap v274: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:11.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:11 smithi120 ceph-mon[32244]: pgmap v274: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:11 smithi170 ceph-mon[32697]: pgmap v274: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-26T18:33:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:13 smithi037 ceph-mon[29171]: pgmap v275: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:13 smithi120 ceph-mon[32244]: pgmap v275: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:13 smithi170 ceph-mon[32697]: pgmap v275: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:15 smithi037 ceph-mon[29171]: pgmap v276: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:15 smithi037 ceph-mon[29171]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:33:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:15 smithi120 ceph-mon[32244]: pgmap v276: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:15 smithi120 ceph-mon[32244]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:33:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:15 smithi170 ceph-mon[32697]: pgmap v276: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:33:15.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:15 smithi170 ceph-mon[32697]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:33:17.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:17 smithi037 ceph-mon[29171]: pgmap v278: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 107 B/s wr, 6 op/s 2024-04-26T18:33:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:17 smithi120 ceph-mon[32244]: pgmap v278: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 107 B/s wr, 6 op/s 2024-04-26T18:33:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:17 smithi170 ceph-mon[32697]: pgmap v278: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 107 B/s wr, 6 op/s 2024-04-26T18:33:19.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:19 smithi037 ceph-mon[29171]: pgmap v279: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:19.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:19 smithi120 ceph-mon[32244]: pgmap v279: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:19 smithi170 ceph-mon[32697]: pgmap v279: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:21 smithi037 ceph-mon[29171]: pgmap v280: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:21 smithi120 ceph-mon[32244]: pgmap v280: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:21 smithi170 ceph-mon[32697]: pgmap v280: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:33:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:23 smithi037 ceph-mon[29171]: pgmap v281: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:23.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:23 smithi120 ceph-mon[32244]: pgmap v281: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:23 smithi170 ceph-mon[32697]: pgmap v281: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:25.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:25 smithi037 ceph-mon[29171]: pgmap v282: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:25.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:25 smithi120 ceph-mon[32244]: pgmap v282: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:25 smithi170 ceph-mon[32697]: pgmap v282: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:33:27.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:27 smithi037 ceph-mon[29171]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T18:33:27.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:27 smithi120 ceph-mon[32244]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T18:33:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:27 smithi170 ceph-mon[32697]: pgmap v283: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-26T18:33:29.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:29 smithi037 ceph-mon[29171]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:29.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:29 smithi120 ceph-mon[32244]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:29 smithi170 ceph-mon[32697]: pgmap v284: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:31.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:31 smithi037 ceph-mon[29171]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:31.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:31 smithi120 ceph-mon[32244]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:31 smithi170 ceph-mon[32697]: pgmap v285: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:33.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:33 smithi037 ceph-mon[29171]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:33.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:33 smithi120 ceph-mon[32244]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:33 smithi170 ceph-mon[32697]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:33:34.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:34.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:34.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:34.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:33:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:34.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:34.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:34.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:33:35.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:35 smithi037 ceph-mon[29171]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:35.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:35 smithi120 ceph-mon[32244]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:35 smithi170 ceph-mon[32697]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:37.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:37 smithi037 ceph-mon[29171]: pgmap v288: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:37.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:37 smithi120 ceph-mon[32244]: pgmap v288: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:37.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:37 smithi170 ceph-mon[32697]: pgmap v288: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:39.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:39 smithi037 ceph-mon[29171]: pgmap v289: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:39.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:39 smithi120 ceph-mon[32244]: pgmap v289: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:39.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:39 smithi170 ceph-mon[32697]: pgmap v289: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:41.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:41 smithi037 ceph-mon[29171]: pgmap v290: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:41.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:41 smithi120 ceph-mon[32244]: pgmap v290: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:41.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:41 smithi170 ceph-mon[32697]: pgmap v290: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:43.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:43 smithi037 ceph-mon[29171]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:43.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:43 smithi120 ceph-mon[32244]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:43.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:43 smithi170 ceph-mon[32697]: pgmap v291: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:45.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:45 smithi037 ceph-mon[29171]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:45.506 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:45 smithi120 ceph-mon[32244]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:45.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:45 smithi170 ceph-mon[32697]: pgmap v292: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:47.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:47 smithi037 ceph-mon[29171]: pgmap v293: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:47.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:47 smithi120 ceph-mon[32244]: pgmap v293: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:47.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:47 smithi170 ceph-mon[32697]: pgmap v293: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:49.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:49 smithi037 ceph-mon[29171]: pgmap v294: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:49.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:49 smithi120 ceph-mon[32244]: pgmap v294: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:49.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:49 smithi170 ceph-mon[32697]: pgmap v294: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:51.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:51 smithi037 ceph-mon[29171]: pgmap v295: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:51.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:51 smithi120 ceph-mon[32244]: pgmap v295: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:51.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:51 smithi170 ceph-mon[32697]: pgmap v295: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:53.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:53 smithi037 ceph-mon[29171]: pgmap v296: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:53.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:53 smithi120 ceph-mon[32244]: pgmap v296: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:53.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:53 smithi170 ceph-mon[32697]: pgmap v296: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:55.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:55 smithi037 ceph-mon[29171]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:55.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:55 smithi120 ceph-mon[32244]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:55.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:55 smithi170 ceph-mon[32697]: pgmap v297: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:57.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:57 smithi037 ceph-mon[29171]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:57 smithi120 ceph-mon[32244]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:57.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:57 smithi170 ceph-mon[32697]: pgmap v298: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:33:59.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:33:59 smithi037 ceph-mon[29171]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:33:59 smithi120 ceph-mon[32244]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:33:59.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:33:59 smithi170 ceph-mon[32697]: pgmap v299: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:01.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:01 smithi037 ceph-mon[29171]: pgmap v300: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:01.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:01 smithi120 ceph-mon[32244]: pgmap v300: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:01 smithi170 ceph-mon[32697]: pgmap v300: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:03.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:03 smithi037 ceph-mon[29171]: pgmap v301: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:34:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:03 smithi120 ceph-mon[32244]: pgmap v301: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:34:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:03 smithi170 ceph-mon[32697]: pgmap v301: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:34:05.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:05 smithi037 ceph-mon[29171]: pgmap v302: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:34:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:05 smithi120 ceph-mon[32244]: pgmap v302: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:34:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:05 smithi170 ceph-mon[32697]: pgmap v302: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:34:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:07 smithi037 ceph-mon[29171]: pgmap v303: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:34:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:07 smithi037 ceph-mon[29171]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:34:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:07 smithi120 ceph-mon[32244]: pgmap v303: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:34:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:07 smithi120 ceph-mon[32244]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:34:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:07 smithi170 ceph-mon[32697]: pgmap v303: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-26T18:34:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:07 smithi170 ceph-mon[32697]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:34:09.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:09 smithi120 ceph-mon[32244]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:09 smithi170 ceph-mon[32697]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:09.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:09 smithi037 ceph-mon[29171]: pgmap v305: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:11 smithi120 ceph-mon[32244]: pgmap v306: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:11.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:11 smithi170 ceph-mon[32697]: pgmap v306: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:11.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:11 smithi037 ceph-mon[29171]: pgmap v306: 97 pgs: 97 active+clean; 131 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-26T18:34:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:13 smithi120 ceph-mon[32244]: pgmap v307: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:13 smithi170 ceph-mon[32697]: pgmap v307: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:13.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:13 smithi037 ceph-mon[29171]: pgmap v307: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:15 smithi120 ceph-mon[32244]: pgmap v308: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:15 smithi120 ceph-mon[32244]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:34:15.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:15 smithi170 ceph-mon[32697]: pgmap v308: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:15.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:15 smithi170 ceph-mon[32697]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:34:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:15 smithi037 ceph-mon[29171]: pgmap v308: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-26T18:34:15.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:15 smithi037 ceph-mon[29171]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:34:17.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:17 smithi120 ceph-mon[32244]: pgmap v310: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 108 B/s wr, 6 op/s 2024-04-26T18:34:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:17 smithi170 ceph-mon[32697]: pgmap v310: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 108 B/s wr, 6 op/s 2024-04-26T18:34:17.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:17 smithi037 ceph-mon[29171]: pgmap v310: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 108 B/s wr, 6 op/s 2024-04-26T18:34:19.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:19 smithi120 ceph-mon[32244]: pgmap v311: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:19 smithi170 ceph-mon[32697]: pgmap v311: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:19.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:19 smithi037 ceph-mon[29171]: pgmap v311: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:21 smithi120 ceph-mon[32244]: pgmap v312: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:21 smithi170 ceph-mon[32697]: pgmap v312: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:21.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:21 smithi037 ceph-mon[29171]: pgmap v312: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T18:34:22.171 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 -- 172.21.15.37:0/3870310055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e0101830 msgr2=0x7f27e0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 --2- 172.21.15.37:0/3870310055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e0101830 0x7f27e0101cb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009930 tx=0x7f27d402a420 comp rx=0 tx=0).stop 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 -- 172.21.15.37:0/3870310055 shutdown_connections 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 --2- 172.21.15.37:0/3870310055 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e01021f0 0x7f27e010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 --2- 172.21.15.37:0/3870310055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e0101830 0x7f27e0101cb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 --2- 172.21.15.37:0/3870310055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0107840 0x7f27e0107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 -- 172.21.15.37:0/3870310055 >> 172.21.15.37:0/3870310055 conn(0x7f27e00789b0 msgr2=0x7f27e0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 -- 172.21.15.37:0/3870310055 shutdown_connections 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.289+0000 7f27e542a640 1 -- 172.21.15.37:0/3870310055 wait complete. 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 Processor -- start 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 -- start start 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0101830 0x7f27e0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 0x7f27e019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111b50 con 0x7f27e01021f0 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01119d0 con 0x7f27e0107840 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27e542a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0111cd0 con 0x7f27e0101830 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43488/0 (socket says 172.21.15.37:43488) 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27de7fc640 1 -- 172.21.15.37:0/124788551 learned_addr learned my addr 172.21.15.37:0/124788551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27deffd640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0101830 0x7f27e0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.290+0000 7f27df7fe640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 0x7f27e019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 -- 172.21.15.37:0/124788551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0101830 msgr2=0x7f27e0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0101830 0x7f27e0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 -- 172.21.15.37:0/124788551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 msgr2=0x7f27e019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 0x7f27e019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 -- 172.21.15.37:0/124788551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019e250 con 0x7f27e01021f0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27df7fe640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 0x7f27e019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27de7fc640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005ab0 tx=0x7f27d4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d403f070 con 0x7f27e01021f0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d4031d60 con 0x7f27e01021f0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e019e4e0 con 0x7f27e01021f0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4035030 con 0x7f27e01021f0 2024-04-26T18:34:22.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.291+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a5dd0 con 0x7f27e01021f0 2024-04-26T18:34:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.292+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e00506c0 con 0x7f27e01021f0 2024-04-26T18:34:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.293+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27d4042070 con 0x7f27e01021f0 2024-04-26T18:34:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.294+0000 7f27c7fff640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 0x7f27c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.294+0000 7f27deffd640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 0x7f27c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.295+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f27d40f1080 con 0x7f27e01021f0 2024-04-26T18:34:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.295+0000 7f27deffd640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 0x7f27c80a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005fd0 tx=0x7f27d000b390 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.299+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27d40bd6e0 con 0x7f27e01021f0 2024-04-26T18:34:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.509+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f27e0062d70 con 0x7f27c80a3dc0 2024-04-26T18:34:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.517+0000 7f27c7fff640 1 -- 172.21.15.37:0/124788551 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f27e0062d70 con 0x7f27c80a3dc0 2024-04-26T18:34:22.517 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:24:20", "first": "2024-04-26T18:25:00", "last": "2024-04-26T18:34:00", "last_pruned": "2024-04-26T18:34:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 msgr2=0x7f27c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 0x7f27c80a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005fd0 tx=0x7f27d000b390 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 msgr2=0x7f27e01997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005ab0 tx=0x7f27d4003cd0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 shutdown_connections 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f27c80a3dc0 0x7f27c80a6280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27e0107840 0x7f27e019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e01021f0 0x7f27e01997a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 --2- 172.21.15.37:0/124788551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27e0101830 0x7f27e0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 >> 172.21.15.37:0/124788551 conn(0x7f27e00789b0 msgr2=0x7f27e006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 shutdown_connections 2024-04-26T18:34:22.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.519+0000 7f27e542a640 1 -- 172.21.15.37:0/124788551 wait complete. 2024-04-26T18:34:22.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 -- 172.21.15.37:0/588960673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74410cfb0 msgr2=0x7ff74410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 --2- 172.21.15.37:0/588960673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74410cfb0 0x7ff74410f3e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009ca0 tx=0x7ff73c02a950 comp rx=0 tx=0).stop 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 -- 172.21.15.37:0/588960673 shutdown_connections 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 --2- 172.21.15.37:0/588960673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff744104500 0x7ff74410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 --2- 172.21.15.37:0/588960673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff74410cfb0 0x7ff74410f3e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 --2- 172.21.15.37:0/588960673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 0x7ff744103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 -- 172.21.15.37:0/588960673 >> 172.21.15.37:0/588960673 conn(0x7ff744077bb0 msgr2=0x7ff744077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 -- 172.21.15.37:0/588960673 shutdown_connections 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.633+0000 7ff74c722640 1 -- 172.21.15.37:0/588960673 wait complete. 2024-04-26T18:34:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 Processor -- start 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 -- start start 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 0x7ff7441990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff74410cfb0 0x7ff74419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff744112bf0 con 0x7ff744104500 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff744112a70 con 0x7ff74410cfb0 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74c722640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff744112d70 con 0x7ff744103be0 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43502/0 (socket says 172.21.15.37:43502) 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 -- 172.21.15.37:0/1082104345 learned_addr learned my addr 172.21.15.37:0/1082104345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74acbf640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff74410cfb0 0x7ff74419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff74a4be640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 0x7ff7441990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 -- 172.21.15.37:0/1082104345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 msgr2=0x7ff7441990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 0x7ff7441990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 -- 172.21.15.37:0/1082104345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff74410cfb0 msgr2=0x7ff74419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff74410cfb0 0x7ff74419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.634+0000 7ff749cbd640 1 -- 172.21.15.37:0/1082104345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74419e0a0 con 0x7ff744104500 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff749cbd640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e9f0 tx=0x7ff73400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73400cd80 con 0x7ff744104500 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7340108e0 con 0x7ff744104500 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74419e390 con 0x7ff744104500 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734018980 con 0x7ff744104500 2024-04-26T18:34:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.635+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a5c20 con 0x7ff744104500 2024-04-26T18:34:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.636+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7440506c0 con 0x7ff744104500 2024-04-26T18:34:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.637+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff734010430 con 0x7ff744104500 2024-04-26T18:34:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.638+0000 7ff73b7fe640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 0x7ff7300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.638+0000 7ff74a4be640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 0x7ff7300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.638+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7ff7340d0b30 con 0x7ff744104500 2024-04-26T18:34:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.638+0000 7ff74a4be640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 0x7ff7300a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff740009bd0 tx=0x7ff740005f90 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.643+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff73409cae0 con 0x7ff744104500 2024-04-26T18:34:22.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.850+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 --> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff7440ffea0 con 0x7ff7300a3dc0 2024-04-26T18:34:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.858+0000 7ff73b7fe640 1 -- 172.21.15.37:0/1082104345 <== mgr.24820 v2:172.21.15.37:6800/1702580825 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff7440ffea0 con 0x7ff7300a3dc0 2024-04-26T18:34:22.858 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-26T18:34:22.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 msgr2=0x7ff7300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 0x7ff7300a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff740009bd0 tx=0x7ff740005f90 comp rx=0 tx=0).stop 2024-04-26T18:34:22.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 msgr2=0x7ff7441995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e9f0 tx=0x7ff73400eeb0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 shutdown_connections 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7ff7300a3dc0 0x7ff7300a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff74410cfb0 0x7ff74419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff744104500 0x7ff7441995f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 --2- 172.21.15.37:0/1082104345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff744103be0 0x7ff7441990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 >> 172.21.15.37:0/1082104345 conn(0x7ff744077bb0 msgr2=0x7ff7440ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 shutdown_connections 2024-04-26T18:34:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.860+0000 7ff74c722640 1 -- 172.21.15.37:0/1082104345 wait complete. 2024-04-26T18:34:22.871 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T18:34:22.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T18:34:22.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1014705070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 msgr2=0x7f8cbc10e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1014705070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc10e6e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009cd0 tx=0x7f8cb402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1014705070 shutdown_connections 2024-04-26T18:34:22.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1014705070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc101810 0x7f8cbc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1014705070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc10e6e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1014705070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc107820 0x7f8cbc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1014705070 >> 172.21.15.37:0/1014705070 conn(0x7f8cbc078930 msgr2=0x7f8cbc078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1014705070 shutdown_connections 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1014705070 wait complete. 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.986+0000 7f8cc0e74640 1 Processor -- start 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 -- start start 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 0x7f8cbc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 0x7f8cbc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111ae0 con 0x7f8cbc1021d0 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111960 con 0x7f8cbc107820 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cc0e74640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc111c60 con 0x7f8cbc101810 2024-04-26T18:34:22.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43526/0 (socket says 172.21.15.37:43526) 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 -- 172.21.15.37:0/1753746423 learned_addr learned my addr 172.21.15.37:0/1753746423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cba59c640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 0x7f8cbc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cbad9d640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 0x7f8cbc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 -- 172.21.15.37:0/1753746423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 msgr2=0x7f8cbc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 0x7f8cbc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 -- 172.21.15.37:0/1753746423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 msgr2=0x7f8cbc19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 0x7f8cbc19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 -- 172.21.15.37:0/1753746423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc19e0e0 con 0x7f8cbc1021d0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cba59c640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 0x7f8cbc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cbad9d640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 0x7f8cbc19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb3fff640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.987+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400cd80 con 0x7f8cbc1021d0 2024-04-26T18:34:22.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.988+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc19e3d0 con 0x7f8cbc1021d0 2024-04-26T18:34:22.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.988+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca40108e0 con 0x7f8cbc1021d0 2024-04-26T18:34:22.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.988+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4018980 con 0x7f8cbc1021d0 2024-04-26T18:34:22.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.988+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a5c60 con 0x7f8cbc1021d0 2024-04-26T18:34:22.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.989+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc069750 con 0x7f8cbc1021d0 2024-04-26T18:34:22.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.990+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ca4010430 con 0x7f8cbc1021d0 2024-04-26T18:34:22.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.991+0000 7f8cb37fe640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 0x7f8ca80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:22.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.991+0000 7f8cba59c640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 0x7f8ca80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:22.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.991+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f8ca40d1110 con 0x7f8cbc1021d0 2024-04-26T18:34:22.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.991+0000 7f8cba59c640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 0x7f8ca80a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005fd0 tx=0x7f8cac00b5d0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:22.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:22.995+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ca409d770 con 0x7f8cbc1021d0 2024-04-26T18:34:23.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.183+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cbc19a440 con 0x7f8cbc1021d0 2024-04-26T18:34:23.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.184+0000 7f8cb37fe640 1 -- 172.21.15.37:0/1753746423 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+23194 (secure 0 0 0) 0x7f8cbc19a440 con 0x7f8cbc1021d0 2024-04-26T18:34:23.188 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:34:23.191 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":102,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:34:15.127932+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3120809922":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1405864097":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3994756262":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/2586849717":"2024-04-27T18:24:19.519750+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:6800/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 msgr2=0x7f8ca80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 0x7f8ca80a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005fd0 tx=0x7f8cac00b5d0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 msgr2=0x7f8cbc199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 shutdown_connections 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f8ca80a3d20 0x7f8ca80a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cbc107820 0x7f8cbc19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cbc1021d0 0x7f8cbc199630 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 --2- 172.21.15.37:0/1753746423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cbc101810 0x7f8cbc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 >> 172.21.15.37:0/1753746423 conn(0x7f8cbc078930 msgr2=0x7f8cbc069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 shutdown_connections 2024-04-26T18:34:23.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.187+0000 7f8cc0e74640 1 -- 172.21.15.37:0/1753746423 wait complete. 2024-04-26T18:34:23.199 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:16:20.665733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T18:21:04.549606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T18:21:07.522605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T18:34:23.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T18:34:23.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.306+0000 7fd02afda640 1 -- 172.21.15.37:0/3405229436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024101800 msgr2=0x7fd024101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.306+0000 7fd02afda640 1 --2- 172.21.15.37:0/3405229436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024101800 0x7fd024101c80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd010009930 tx=0x7fd01002a420 comp rx=0 tx=0).stop 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- 172.21.15.37:0/3405229436 shutdown_connections 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- 172.21.15.37:0/3405229436 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 0x7fd02410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- 172.21.15.37:0/3405229436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024101800 0x7fd024101c80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- 172.21.15.37:0/3405229436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024107810 0x7fd024107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- 172.21.15.37:0/3405229436 >> 172.21.15.37:0/3405229436 conn(0x7fd024078930 msgr2=0x7fd024078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- 172.21.15.37:0/3405229436 shutdown_connections 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- 172.21.15.37:0/3405229436 wait complete. 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 Processor -- start 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- start start 2024-04-26T18:34:23.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024101800 0x7fd02410f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 0x7fd02410f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd024112bd0 con 0x7fd024107810 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd024112a50 con 0x7fd0241021c0 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.307+0000 7fd02afda640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd024112d50 con 0x7fd024101800 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43542/0 (socket says 172.21.15.37:43542) 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 -- 172.21.15.37:0/1343424191 learned_addr learned my addr 172.21.15.37:0/1343424191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd028d76640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024101800 0x7fd02410f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd023fff640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 0x7fd02410f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 -- 172.21.15.37:0/1343424191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024101800 msgr2=0x7fd02410f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024101800 0x7fd02410f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 -- 172.21.15.37:0/1343424191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 msgr2=0x7fd02410f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 0x7fd02410f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 -- 172.21.15.37:0/1343424191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0241a7650 con 0x7fd024107810 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd029577640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb10 tx=0x7fd01c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c004b20 con 0x7fd024107810 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c004cc0 con 0x7fd024107810 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0241a7940 con 0x7fd024107810 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c010670 con 0x7fd024107810 2024-04-26T18:34:23.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.308+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0241a7e60 con 0x7fd024107810 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.310+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd024069750 con 0x7fd024107810 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.311+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd01c005020 con 0x7fd024107810 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.311+0000 7fd021ffb640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 0x7fd0140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.311+0000 7fd028d76640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 0x7fd0140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.311+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fd01c0d0e30 con 0x7fd024107810 2024-04-26T18:34:23.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.312+0000 7fd028d76640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 0x7fd0140a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd018005fd0 tx=0x7fd01800b390 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.316+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd01c09d490 con 0x7fd024107810 2024-04-26T18:34:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:23 smithi037 ceph-mon[29171]: pgmap v313: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:23.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:23 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1753746423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.503+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd024109d90 con 0x7fd024107810 2024-04-26T18:34:23.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.503+0000 7fd021ffb640 1 -- 172.21.15.37:0/1343424191 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7fd01c09cd90 con 0x7fd024107810 2024-04-26T18:34:23.504 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T18:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.505+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 msgr2=0x7fd0140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.505+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 0x7fd0140a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd018005fd0 tx=0x7fd01800b390 comp rx=0 tx=0).stop 2024-04-26T18:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.505+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 msgr2=0x7fd02410fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.505+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb10 tx=0x7fd01c00c2d0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 shutdown_connections 2024-04-26T18:34:23.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd0140a3da0 0x7fd0140a6260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0241021c0 0x7fd02410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024107810 0x7fd02410fbe0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 --2- 172.21.15.37:0/1343424191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd024101800 0x7fd02410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 >> 172.21.15.37:0/1343424191 conn(0x7fd024078930 msgr2=0x7fd024104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 shutdown_connections 2024-04-26T18:34:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.506+0000 7fd02afda640 1 -- 172.21.15.37:0/1343424191 wait complete. 2024-04-26T18:34:23.517 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T18:34:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:23 smithi170 ceph-mon[32697]: pgmap v313: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:23.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:23 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1753746423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 -- 172.21.15.37:0/128998239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1021d0 msgr2=0x7efc6c10e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 --2- 172.21.15.37:0/128998239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1021d0 0x7efc6c10e6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc64009ca0 tx=0x7efc6402a8c0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 -- 172.21.15.37:0/128998239 shutdown_connections 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 --2- 172.21.15.37:0/128998239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1021d0 0x7efc6c10e6e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 --2- 172.21.15.37:0/128998239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c101810 0x7efc6c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 --2- 172.21.15.37:0/128998239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 -- 172.21.15.37:0/128998239 >> 172.21.15.37:0/128998239 conn(0x7efc6c078890 msgr2=0x7efc6c078ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 -- 172.21.15.37:0/128998239 shutdown_connections 2024-04-26T18:34:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.621+0000 7efc70faf640 1 -- 172.21.15.37:0/128998239 wait complete. 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 Processor -- start 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 -- start start 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c1021d0 0x7efc6c10f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c10f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c112be0 con 0x7efc6c101810 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c112a60 con 0x7efc6c107820 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc70faf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c112d60 con 0x7efc6c1021d0 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43552/0 (socket says 172.21.15.37:43552) 2024-04-26T18:34:23.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc6a59c640 1 -- 172.21.15.37:0/352531547 learned_addr learned my addr 172.21.15.37:0/352531547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc6ad9d640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c10f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.622+0000 7efc69d9b640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c1021d0 0x7efc6c10f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 -- 172.21.15.37:0/352531547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c1021d0 msgr2=0x7efc6c10f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c1021d0 0x7efc6c10f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 -- 172.21.15.37:0/352531547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 msgr2=0x7efc6c10f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c10f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 -- 172.21.15.37:0/352531547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c1a74a0 con 0x7efc6c101810 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6ad9d640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc6a59c640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00e9f0 tx=0x7efc5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00cd80 con 0x7efc6c101810 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c1a7790 con 0x7efc6c101810 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc5c0108e0 con 0x7efc6c101810 2024-04-26T18:34:23.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c018980 con 0x7efc6c101810 2024-04-26T18:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.623+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a7d30 con 0x7efc6c101810 2024-04-26T18:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.624+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c069ba0 con 0x7efc6c101810 2024-04-26T18:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.625+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc5c010430 con 0x7efc6c101810 2024-04-26T18:34:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.626+0000 7efc637fe640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 0x7efc580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.626+0000 7efc69d9b640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 0x7efc580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.626+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7efc5c0d10f0 con 0x7efc6c101810 2024-04-26T18:34:23.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.626+0000 7efc69d9b640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 0x7efc580a6210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc54005e10 tx=0x7efc54005da0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.630+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc5c09d6f0 con 0x7efc6c101810 2024-04-26T18:34:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:23 smithi120 ceph-mon[32244]: pgmap v313: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:23.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:23 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1753746423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:34:23.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.818+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efc6c109d80 con 0x7efc6c101810 2024-04-26T18:34:23.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.819+0000 7efc637fe640 1 -- 172.21.15.37:0/352531547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7efc5c0bd0a0 con 0x7efc6c101810 2024-04-26T18:34:23.819 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T18:34:23.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 msgr2=0x7efc580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 0x7efc580a6210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc54005e10 tx=0x7efc54005da0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 msgr2=0x7efc6c10ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00e9f0 tx=0x7efc5c00eeb0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 shutdown_connections 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7efc580a3d50 0x7efc580a6210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c107820 0x7efc6c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc6c1021d0 0x7efc6c10f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 --2- 172.21.15.37:0/352531547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c101810 0x7efc6c10ee70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 >> 172.21.15.37:0/352531547 conn(0x7efc6c078890 msgr2=0x7efc6c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 shutdown_connections 2024-04-26T18:34:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.821+0000 7efc70faf640 1 -- 172.21.15.37:0/352531547 wait complete. 2024-04-26T18:34:23.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T18:34:23.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 -- 172.21.15.37:0/1902259418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f289010f2e0 msgr2=0x7f28901116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 --2- 172.21.15.37:0/1902259418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f289010f2e0 0x7f28901116d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ca0 tx=0x7f288802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 -- 172.21.15.37:0/1902259418 shutdown_connections 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 --2- 172.21.15.37:0/1902259418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f289010f2e0 0x7f28901116d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 --2- 172.21.15.37:0/1902259418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2890102050 0x7f289010eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 --2- 172.21.15.37:0/1902259418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2890101730 0x7f2890101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 -- 172.21.15.37:0/1902259418 >> 172.21.15.37:0/1902259418 conn(0x7f2890078930 msgr2=0x7f2890078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.936+0000 7f289513f640 1 -- 172.21.15.37:0/1902259418 shutdown_connections 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.937+0000 7f289513f640 1 -- 172.21.15.37:0/1902259418 wait complete. 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.937+0000 7f289513f640 1 Processor -- start 2024-04-26T18:34:23.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.937+0000 7f289513f640 1 -- start start 2024-04-26T18:34:23.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2890101730 0x7f289019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f289010f2e0 0x7f289019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2890114dc0 con 0x7f2890102050 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2890114c40 con 0x7f289010f2e0 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2890114f40 con 0x7f2890101730 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43570/0 (socket says 172.21.15.37:43570) 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 -- 172.21.15.37:0/267936917 learned_addr learned my addr 172.21.15.37:0/267936917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288f59e640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f289010f2e0 0x7f289019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 -- 172.21.15.37:0/267936917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2890101730 msgr2=0x7f289019b2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2890101730 0x7f289019b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 -- 172.21.15.37:0/267936917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f289010f2e0 msgr2=0x7f289019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:23.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f289010f2e0 0x7f289019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 -- 172.21.15.37:0/267936917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28901a02d0 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f288e59c640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9f0 tx=0x7f288400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400cd80 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901a05c0 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28840108e0 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.938+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884018980 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.939+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901a7e50 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.939+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2890069750 con 0x7f2890102050 2024-04-26T18:34:23.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.941+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2884010430 con 0x7f2890102050 2024-04-26T18:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.941+0000 7f2877fff640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 0x7f28780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.941+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f28840d1000 con 0x7f2890102050 2024-04-26T18:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.941+0000 7f288ed9d640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 0x7f28780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:23.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.942+0000 7f288ed9d640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 0x7f28780a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f288000b990 tx=0x7f288000b370 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:23.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:23.945+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f288409d660 con 0x7f2890102050 2024-04-26T18:34:24.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.129+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f289002dad0 con 0x7f2890102050 2024-04-26T18:34:24.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.130+0000 7f2877fff640 1 -- 172.21.15.37:0/267936917 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f288409cf60 con 0x7f2890102050 2024-04-26T18:34:24.130 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 msgr2=0x7f28780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 0x7f28780a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f288000b990 tx=0x7f288000b370 comp rx=0 tx=0).stop 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 msgr2=0x7f289019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9f0 tx=0x7f288400eeb0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 shutdown_connections 2024-04-26T18:34:24.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f28780a3d20 0x7f28780a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f289010f2e0 0x7f289019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2890102050 0x7f289019b820 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 --2- 172.21.15.37:0/267936917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2890101730 0x7f289019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 >> 172.21.15.37:0/267936917 conn(0x7f2890078930 msgr2=0x7f2890110a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 shutdown_connections 2024-04-26T18:34:24.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.132+0000 7f289513f640 1 -- 172.21.15.37:0/267936917 wait complete. 2024-04-26T18:34:24.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T18:34:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 -- 172.21.15.37:0/2550069353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1248108940 msgr2=0x7f1248108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 --2- 172.21.15.37:0/2550069353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1248108940 0x7f1248108d20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1238009900 tx=0x7f123802a480 comp rx=0 tx=0).stop 2024-04-26T18:34:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 -- 172.21.15.37:0/2550069353 shutdown_connections 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 --2- 172.21.15.37:0/2550069353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248103350 0x7f124810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 --2- 172.21.15.37:0/2550069353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12481029b0 0x7f1248102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 --2- 172.21.15.37:0/2550069353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1248108940 0x7f1248108d20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 -- 172.21.15.37:0/2550069353 >> 172.21.15.37:0/2550069353 conn(0x7f1248079ac0 msgr2=0x7f1248079ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 -- 172.21.15.37:0/2550069353 shutdown_connections 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.244+0000 7f124e9f2640 1 -- 172.21.15.37:0/2550069353 wait complete. 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 Processor -- start 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 -- start start 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 0x7f1248075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 0x7f12480756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1248112bb0 con 0x7f12481029b0 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1248112a30 con 0x7f1248108940 2024-04-26T18:34:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124e9f2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1248112d30 con 0x7f1248103350 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f1247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f1247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43592/0 (socket says 172.21.15.37:43592) 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f1247fff640 1 -- 172.21.15.37:0/1726593325 learned_addr learned my addr 172.21.15.37:0/1726593325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f12477fe640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 0x7f1248075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.245+0000 7f124cf8f640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 0x7f12480756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 -- 172.21.15.37:0/1726593325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 msgr2=0x7f1248075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 0x7f1248075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 -- 172.21.15.37:0/1726593325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 msgr2=0x7f12480756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 0x7f12480756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 -- 172.21.15.37:0/1726593325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12481a76a0 con 0x7f12481029b0 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f12477fe640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 0x7f1248075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f124cf8f640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 0x7f12480756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:24.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f1247fff640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1238005940 tx=0x7f1238004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123803f030 con 0x7f12481029b0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1238003aa0 con 0x7f12481029b0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238035070 con 0x7f12481029b0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481a7930 con 0x7f12481029b0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.246+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a7e20 con 0x7f12481029b0 2024-04-26T18:34:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.247+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1248069ba0 con 0x7f12481029b0 2024-04-26T18:34:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.248+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1238042030 con 0x7f12481029b0 2024-04-26T18:34:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.249+0000 7f12457fa640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 0x7f12300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.249+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f12380f0f50 con 0x7f12481029b0 2024-04-26T18:34:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.249+0000 7f12477fe640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 0x7f12300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.249+0000 7f12477fe640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 0x7f12300a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1248076670 tx=0x7f123c005f90 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.253+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12380bd550 con 0x7f12481029b0 2024-04-26T18:34:24.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:24 smithi037 ceph-mon[29171]: from='client.16083 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:24 smithi037 ceph-mon[29171]: from='client.16089 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:24 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1343424191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:34:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:24 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/352531547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:34:24.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:24 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/267936917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:34:24.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.493+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f124810ae10 con 0x7f12481029b0 2024-04-26T18:34:24.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.501+0000 7f12457fa640 1 -- 172.21.15.37:0/1726593325 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f12380bce50 con 0x7f12481029b0 2024-04-26T18:34:24.512 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:34:24.512 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:34:24.512 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T18:34:24.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.512 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 24820, 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1702580825 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1702580825 2024-04-26T18:34:24.513 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/1702580825", 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-04-26T18:24:19.519969+0000", 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24817, 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T18:34:24.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.519 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T18:34:24.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:34:24.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.543 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T18:34:24.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.546 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T18:34:24.547 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:24.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.552 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T18:34:24.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.554 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T18:34:24.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T18:34:24.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T18:34:24.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T18:34:24.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T18:34:24.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:34:24.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T18:34:24.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T18:34:24.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.568 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:34:24.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:34:24.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T18:34:24.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.595 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.603 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.604 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.605 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:34:24.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:34:24.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.618 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.619 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.623 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T18:34:24.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:24.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:34:24.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:34:24.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:34:24.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:34:24.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:34:24.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:34:24.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:34:24.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:34:24.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:34:24.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:34:24.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.658 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.659 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.660 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.661 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.662 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.663 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.665 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:34:24.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.678 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.681 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.687 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T18:34:24.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:24.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T18:34:24.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.697 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.699 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.700 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T18:34:24.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:24.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.708 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.711 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.712 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.714 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.715 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.717 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:34:24.721 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T18:34:24.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.737 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T18:34:24.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:24.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.741 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.744 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.762 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:34:24.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.764 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:34:24.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T18:34:24.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.782 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.783 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.786 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T18:34:24.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.797 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.799 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.800 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.802 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T18:34:24.803 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.804 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.805 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.808 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.814 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.816 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.818 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T18:34:24.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.829 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.832 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:24.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T18:34:24.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.839 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.840 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.843 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:34:24.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.847 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:34:24.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.856 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.859 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:34:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.862 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.865 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.877 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.880 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.883 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.884 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.887 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:34:24.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.892 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.895 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.897 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.902 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T18:34:24.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.912 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:34:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T18:34:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.931 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.935 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:34:24.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.946 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.953 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T18:34:24.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T18:34:24.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T18:34:24.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T18:34:24.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T18:34:24.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.977 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T18:34:24.979 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.980 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.983 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:34:24.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T18:34:24.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.993 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.995 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:24.998 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:24.999 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24821, 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T18:34:25.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T18:34:25.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T18:34:25.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T18:34:25.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T18:34:25.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T18:34:25.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:34:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T18:34:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T18:34:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:34:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.038 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T18:34:25.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:34:25.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.050 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:34:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.055 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T18:34:25.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:34:25.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T18:34:25.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.067 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T18:34:25.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T18:34:25.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:34:25.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:34:25.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T18:34:25.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.087 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:34:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.090 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.092 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.095 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.096 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.098 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:25.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:34:25.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.110 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T18:34:25.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.117 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:34:25.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.131 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:34:25.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:34:25.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.146 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:34:25.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:34:25.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:34:25.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:34:25.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.168 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:34:25.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:34:25.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.180 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:34:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.187 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:25.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.195 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.199 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.202 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.208 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:25.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.220 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.223 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T18:34:25.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.244 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:25.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T18:34:25.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T18:34:25.256 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.257 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.260 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.269 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T18:34:25.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.273 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.278 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:34:25.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.280 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:34:25.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T18:34:25.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.299 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.302 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T18:34:25.304 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.305 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.306 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.308 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.313 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.316 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.321 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.324 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.327 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.334 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.335 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.341 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:25.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.346 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.349 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:34:25.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T18:34:25.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:34:25.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.365 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:34:25.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.367 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:34:25.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:25.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.400 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.403 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.412 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T18:34:25.414 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.416 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.417 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.424 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T18:34:25.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.431 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.442 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.447 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.451 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.471 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T18:34:25.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.476 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:34:25.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T18:34:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.507 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.509 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T18:34:25.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T18:34:25.516 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T18:34:25.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-04-26T18:34:25.530 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.531 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:34:25.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.534 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.537 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T18:34:25.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.545 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:34:25.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T18:34:25.550 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T18:34:25.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:34:25.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.555 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:34:25.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T18:34:25.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T18:34:25.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.561 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T18:34:25.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.571 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:34:25.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T18:34:25.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T18:34:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:34:25.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T18:34:25.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:34:25.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T18:34:25.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.590 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T18:34:25.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:34:25.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:34:25.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.597 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T18:34:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T18:34:25.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:34:25.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T18:34:25.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T18:34:25.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.618 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:34:25.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:34:25.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:34:25.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T18:34:25.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T18:34:25.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.639 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:34:25.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T18:34:25.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.647 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.655 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T18:34:25.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:34:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:34:25.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.670 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:34:25.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.677 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:34:25.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:34:25.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:34:25.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:34:25.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.700 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T18:34:25.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:34:25.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:34:25.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T18:34:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T18:34:25.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.722 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T18:34:25.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T18:34:25.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T18:34:25.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.731 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:25.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T18:34:25.735 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.740 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.742 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T18:34:25.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.748 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:34:25.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.752 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.755 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T18:34:25.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.759 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T18:34:25.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:34:25.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.772 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T18:34:25.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.775 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:25.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.782 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.783 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.785 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.789 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.794 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:34:25.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.800 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.805 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T18:34:25.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.816 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.817 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.819 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.820 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.830 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.835 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.837 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.838 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.842 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T18:34:25.844 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.846 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.848 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.850 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.863 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T18:34:25.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.875 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.878 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T18:34:25.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.884 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.887 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.889 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T18:34:25.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.897 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.905 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T18:34:25.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.915 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.918 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.923 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.924 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.927 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.932 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T18:34:25.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T18:34:25.937 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.946 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T18:34:25.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T18:34:25.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T18:34:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.957 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T18:34:25.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.966 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.970 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.976 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:25.986 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:25.988 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:25.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.002 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.013 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T18:34:26.015 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.029 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T18:34:26.034 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T18:34:26.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T18:34:26.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T18:34:26.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.041 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T18:34:26.042 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.043 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T18:34:26.044 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T18:34:26.045 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 93, 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2422424977 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3365309780 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3517686524 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2526496724 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2305406551 2024-04-26T18:34:26.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 624377212 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T18:34:26.050 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:34:26.052 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:24 smithi120 ceph-mon[32244]: from='client.16083 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:26.052 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:24 smithi120 ceph-mon[32244]: from='client.16089 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:24 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1343424191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:24 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/352531547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:24 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/267936917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:25 smithi120 ceph-mon[32244]: pgmap v314: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:25 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1726593325' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:24 smithi170 ceph-mon[32697]: from='client.16083 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:24 smithi170 ceph-mon[32697]: from='client.16089 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:24 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1343424191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:24 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/352531547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:24 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/267936917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:25 smithi170 ceph-mon[32697]: pgmap v314: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:26.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:25 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1726593325' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 msgr2=0x7f12300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 0x7f12300a6210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1248076670 tx=0x7f123c005f90 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 msgr2=0x7f1248074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1238005940 tx=0x7f1238004270 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 shutdown_connections 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f12300a3d50 0x7f12300a6210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1248108940 0x7f12480756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1248103350 0x7f1248075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 --2- 172.21.15.37:0/1726593325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12481029b0 0x7f1248074c30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 >> 172.21.15.37:0/1726593325 conn(0x7f1248079ac0 msgr2=0x7f12481018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:26.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 shutdown_connections 2024-04-26T18:34:26.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:24.507+0000 7f124e9f2640 1 -- 172.21.15.37:0/1726593325 wait complete. 2024-04-26T18:34:26.060 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T18:34:26.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T18:34:26.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:25 smithi037 ceph-mon[29171]: pgmap v314: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-26T18:34:26.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:25 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1726593325' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:34:26.321 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:27.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:27 smithi037 ceph-mon[29171]: pgmap v315: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-26T18:34:27.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:27 smithi170 ceph-mon[32697]: pgmap v315: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-26T18:34:27.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:27 smithi120 ceph-mon[32244]: pgmap v315: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-26T18:34:28.219 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:34:28.219 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001519242, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000354419, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:28.220 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 167, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:28.221 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186592, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.014281808, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009912164, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002095402, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 164, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014479622, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000088290 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.222 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186592 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 234, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 221, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1595200, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 134912, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:28.223 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.042771295, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013533790, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.033577986, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004245085, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 221, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.010896239, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000049304 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:28.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1595200, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 134912 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:28.225 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:28.226 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:28.227 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148771, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 32, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:28.228 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T18:34:28.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 232, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1559707, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 232, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 232, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1559707, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:28.230 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:28.231 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:28.231 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:34:28.983 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-26T18:34:28.984 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-26T18:34:28.984 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T18:34:28.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T18:34:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:29.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:29 smithi170 ceph-mon[32697]: pgmap v316: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:29.593 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:29 smithi037 ceph-mon[29171]: pgmap v316: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:29.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:29 smithi120 ceph-mon[32244]: pgmap v316: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:30.872 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001406622, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000271654, 2024-04-26T18:34:30.873 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 173, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 187648, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.014166647, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009751516, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002079153, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 170, 2024-04-26T18:34:30.875 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014454986, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000085029 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 187648 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 240, 2024-04-26T18:34:30.876 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 227, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1596832, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 138752, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.045924883, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.015211358, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.035633461, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004502620, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 227, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012396574, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000054610 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.877 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1596832, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 138752 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.878 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:30.879 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:30.880 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148731, 2024-04-26T18:34:30.881 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:30.882 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T18:34:30.883 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 238, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1560487, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 238, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 238, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1560487, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:30.884 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:30.885 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:34:31.499 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T18:34:31.500 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-26T18:34:31.500 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T18:34:31.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T18:34:31.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:31 smithi037 ceph-mon[29171]: pgmap v317: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:31.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:31 smithi170 ceph-mon[32697]: pgmap v317: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:31.750 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:31.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:31 smithi120 ceph-mon[32244]: pgmap v317: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:33.406 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:34:33.406 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3040, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001701031, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000037227, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000618800, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-26T18:34:33.407 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000703029, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000234343 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 233, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 222, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1592976, 2024-04-26T18:34:33.408 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 136384, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.045746899, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.014382443, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.035925886, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004440722, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 221, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.009911768, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000044849 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:33.409 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1592976, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 136384 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 169, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186944, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.013158428, 2024-04-26T18:34:33.410 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009065663, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001873175, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 166, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013610555, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000081991 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:33.411 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186944 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:33.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:33.413 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148794, 2024-04-26T18:34:33.414 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 32, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T18:34:33.415 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 235, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1560097, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 235, 2024-04-26T18:34:33.416 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 235, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1560097, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:33.417 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:34:33.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:33 smithi170 ceph-mon[32697]: pgmap v318: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:33.718 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:33 smithi037 ceph-mon[29171]: pgmap v318: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:33.753 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:33 smithi120 ceph-mon[32244]: pgmap v318: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:34.083 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-26T18:34:34.084 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-26T18:34:34.084 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T18:34:34.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T18:34:34.396 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:34.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:34 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:34.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:34 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:34.752 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:34 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:35.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:35 smithi037 ceph-mon[29171]: pgmap v319: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:35.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:35 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:35 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:35 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.569 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:35 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:35 smithi170 ceph-mon[32697]: pgmap v319: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:35.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:35 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:35 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:35 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:35 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:35 smithi120 ceph-mon[32244]: pgmap v319: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T18:34:35.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:35 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:35 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:35 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:35.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:35 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 239, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 228, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1594608, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 140224, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:36.137 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.046715496, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.015168919, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.035639040, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004549004, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 227, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011353127, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000050013 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:36.138 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1594608, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 140224 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 174, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 187824, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015395971, 2024-04-26T18:34:36.139 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010790609, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002085100, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 171, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012321961, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000072058 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 187824 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3040, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000673346, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000040888, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000625813, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:36.141 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000708340, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000236113 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.142 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:36.143 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:36.144 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:36.145 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148754, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:36.146 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T18:34:36.147 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 241, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1560877, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 241, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 241, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1560877, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T18:34:36.148 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T18:34:36.718 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T18:34:36.719 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-26T18:34:36.719 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T18:34:36.719 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T18:34:36.962 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: pgmap v320: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:36.962 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:36.962 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:36.962 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:36.962 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:36.963 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:36.963 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:36.963 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:36.963 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:36 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:36.964 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: pgmap v320: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.004 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:36 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: pgmap v320: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:37.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:36 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:34:38.164 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:37 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:37 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-26T18:34:38.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi170 to 184791756: error parsing value: Value '184791756' is below minimum 939524096 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi120 to 184771686: error parsing value: Value '184771686' is below minimum 939524096 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:37 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:38.711 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-26T18:34:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 33198, 2024-04-26T18:34:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 33240, 2024-04-26T18:34:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 8657276, 2024-04-26T18:34:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 62104095, 2024-04-26T18:34:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 17, 2024-04-26T18:34:38.712 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 1.977306373, 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.668359804, 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.537723381, 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.667778948, 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:38.713 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 33242, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.736609174, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000052241 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4179, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000726331, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000173 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 350000, 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 88224 2024-04-26T18:34:38.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 18953, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 18898, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 6406197, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 33243133, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 87, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 1.127729001, 2024-04-26T18:34:38.715 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.348156501, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.326492086, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.381987226, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 18896, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.297810812, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000068681 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 359, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000052443, 2024-04-26T18:34:38.716 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000146 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1773328, 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 489552 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 41677, 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 75009, 2024-04-26T18:34:38.717 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 14677836, 2024-04-26T18:34:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 36003050, 2024-04-26T18:34:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 11, 2024-04-26T18:34:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-04-26T18:34:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 2.785262267, 2024-04-26T18:34:38.718 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 1.134003700, 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 1.381830333, 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.356801995, 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 75004, 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2.597835500, 2024-04-26T18:34:38.719 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000034635 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 72892, 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.011837582, 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000162 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 44832, 2024-04-26T18:34:38.720 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 159744 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 87280, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 16.614364979, 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000190357 2024-04-26T18:34:38.721 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 144, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.006145113, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000042674 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 11043, 2024-04-26T18:34:38.722 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 10888, 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 10888, 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 56.431816411, 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005182936 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.723 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 155, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 250, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 2, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 109, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 13, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 85, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 11, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 22, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 1399, 2024-04-26T18:34:38.724 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 2, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 2, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 6, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 12514, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 11870, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 152, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 40, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 3, 2024-04-26T18:34:38.725 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 19, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 1299, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-04-26T18:34:38.726 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 234, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 6, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:38.727 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 8, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 8562, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 9409, 2024-04-26T18:34:38.728 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 7847, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 8641, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 7399, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 21, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 9153 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 18, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 392, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 216, 2024-04-26T18:34:38.729 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 211, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 57, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 6, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 57, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 48, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 3, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-04-26T18:34:38.730 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 4, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 179, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 19, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 2, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:38.731 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 26058, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 26011, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 26011, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 47, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 89, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 60, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 59, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 59, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 1, 2024-04-26T18:34:38.732 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 1, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 2, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 88092917, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 88187108, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 24795, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 38.893641318, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001568608 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-04-26T18:34:38.733 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 71, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 1164, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 1093, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 32, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 256, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 224, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 111, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 1594, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 1483, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 2, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 1111, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 1109, 2024-04-26T18:34:38.734 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 72828, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 231800 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 29322, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 1376, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 11043, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 55, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 3329, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1024, 2024-04-26T18:34:38.735 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3.045537571, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002974157 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 332, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.502983659, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.004527059 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-04-26T18:34:38.736 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.008140994, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002713664 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 84, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.306523493, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003649089 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.737 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.738 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-04-26T18:34:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 10, 2024-04-26T18:34:38.740 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002406103, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000240610 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 46, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 20.438066779, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.444305799 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.741 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 10, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.056571469, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005657146 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 303, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.067396464, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000222430 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 13, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.019716279, 2024-04-26T18:34:38.742 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001516636 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 20, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.051549168, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002577458 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.743 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 6, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.047945177, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.007990862 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.025003627, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.006250906 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:38.744 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8982, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 30.770544290, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003425800 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.021207857, 2024-04-26T18:34:38.746 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002650982 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000677161, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000677161 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 42, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.067546320, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001608245 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:38.747 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 6, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 4, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 2, 2024-04-26T18:34:38.748 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 299, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 149, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 596, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 360, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 360, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:38.749 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:38.750 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:38.751 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 385146, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 164, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:38.752 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 332238, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 3139, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:38.753 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 63108, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 85159043, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 32, 2024-04-26T18:34:38.754 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 63108, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 63108, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 211.588187448, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003352795 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 63108, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 64500 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 63108, 2024-04-26T18:34:38.755 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 9, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 63099, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 178, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 452, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 9, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 24804, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 126, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 2, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 627, 2024-04-26T18:34:38.756 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 905, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:38.757 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 36701, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-04-26T18:34:38.758 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-04-26T18:34:38.759 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 14, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-04-26T18:34:38.760 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 650, 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 46 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-04-26T18:34:38.761 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-04-26T18:34:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-04-26T18:34:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-04-26T18:34:38.762 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 242 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 34, 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 10, 2024-04-26T18:34:38.763 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 75, 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 48, 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:38.764 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 93824, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 21227776, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 93824, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 93824, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 21227776, 2024-04-26T18:34:38.765 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:38.766 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 63108, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 85164723, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 63108, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 85164723, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:38.767 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 63108, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 63108, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 63108, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 63108, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.768 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 48, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 5680, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 48, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:38.769 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 9, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5680, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x558ced2a0500": { 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:38.770 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 26058, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 83992804, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 26058, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 24795, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 83992804, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:38.771 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: pgmap v321: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:39.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:38 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: pgmap v321: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:39.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:38 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:39.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: pgmap v321: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:39.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:39.327 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:38 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds.exported=19 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=18 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=392 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-26T18:34:39.526 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=2 2024-04-26T18:34:39.526 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T18:34:39.527 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T18:34:39.773 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:34:40.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:39 smithi120 ceph-mon[32244]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:34:40.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:39 smithi120 ceph-mon[32244]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:34:40.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:39 smithi037 ceph-mon[29171]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:34:40.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:39 smithi037 ceph-mon[29171]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:34:40.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:39 smithi170 ceph-mon[32697]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-26T18:34:40.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:39 smithi170 ceph-mon[32697]: Unable to set osd_memory_target on smithi037 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T18:34:41.133 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:40 smithi120 ceph-mon[32244]: pgmap v322: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:41.227 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:40 smithi170 ceph-mon[32697]: pgmap v322: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:41.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:40 smithi037 ceph-mon[29171]: pgmap v322: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1060, 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1040, 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1921744, 2024-04-26T18:34:41.396 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 395372, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.130459065, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.048942952, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.053552656, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.028527226, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1039, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.052419222, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000050451 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.397 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1654960, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 142880 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 8, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 178, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3916, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 462858, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-04-26T18:34:41.398 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 4, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.012224073, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.007361693, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.002876189, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000038123, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 176, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.017083735, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000097066 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.399 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 462160 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1526, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1528, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 461455, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 485275, 2024-04-26T18:34:41.400 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.089194370, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.021369612, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.024759304, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.037465450, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1526, 2024-04-26T18:34:41.401 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.037853986, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000024806 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:41.402 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3926, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.093832396, 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000023900 2024-04-26T18:34:41.403 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-04-26T18:34:41.404 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:41.405 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 27, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 27, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 26, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 3, 2024-04-26T18:34:41.406 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-04-26T18:34:41.407 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:41.408 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-04-26T18:34:41.409 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-04-26T18:34:41.410 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-04-26T18:34:41.411 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 49, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-04-26T18:34:41.412 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 2, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 4267014, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 4266440, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.413 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 49 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 15, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 30, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 15, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 6, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 15, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 9, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 28, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 37, 2024-04-26T18:34:41.414 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 9, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 51016, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215416 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-04-26T18:34:41.415 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.416 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.417 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.418 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.419 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-04-26T18:34:41.420 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-04-26T18:34:41.421 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.422 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:41.423 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.424 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.425 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-04-26T18:34:41.427 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.428 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 5, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 3, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 4, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2964, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:41.429 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:41.430 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:41.431 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 182494, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 59, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:41.432 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 64622, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 470, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-04-26T18:34:41.433 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 2322, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 2322, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2322, 2024-04-26T18:34:41.434 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.286970797, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000554250 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2322, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 2323 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 2322, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 2322, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-04-26T18:34:41.435 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 1498, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 822, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:41.436 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-04-26T18:34:41.437 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-04-26T18:34:41.438 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 14, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 4, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 4, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-04-26T18:34:41.439 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-04-26T18:34:41.440 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 2590, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 2137356, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 2590, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:41.441 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2590, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 2137356, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:41.442 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2322, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 72715, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2322, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 72715, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:41.443 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 2322, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 2322, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2322, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 2322, 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:41.444 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:41.445 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x5651758906e0": { 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:41.446 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-04-26T18:34:41.447 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:41.448 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:34:42.140 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:34:42.141 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-26T18:34:42.141 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T18:34:42.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T18:34:42.376 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:34:43.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:42 smithi037 ceph-mon[29171]: pgmap v323: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:43.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:42 smithi120 ceph-mon[32244]: pgmap v323: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:43.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:42 smithi170 ceph-mon[32697]: pgmap v323: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:43.923 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-26T18:34:43.923 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:43.923 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 5563, 2024-04-26T18:34:43.923 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 3716, 2024-04-26T18:34:43.923 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 5797070, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 1173922, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 9, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.315025771, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.130787808, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.224947399, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.010969467, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3715, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.104197952, 2024-04-26T18:34:43.924 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000028047 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 8502, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001207217, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000141 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1672192, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 469, 2024-04-26T18:34:43.925 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 694, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 134678, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 385752, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 17, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.046786411, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.016131486, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.031896019, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000991354, 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:43.926 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 692, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.028616409, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000041353 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 886, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000145906, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000164 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 7792, 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 154576 2024-04-26T18:34:43.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 42, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 183, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 10998, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 360597, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 10, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 7, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.012103291, 2024-04-26T18:34:43.928 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.005916927, 2024-04-26T18:34:43.929 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.004335252, 2024-04-26T18:34:43.929 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000766857, 2024-04-26T18:34:43.929 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:43.929 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 181, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.030325959, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000167546 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 32, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000006066, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000189 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.930 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 317472 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 117, 2024-04-26T18:34:43.931 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.012177912, 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000104084 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-04-26T18:34:43.932 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 22, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000848794, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000038581 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 260, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 38, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 38, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.031110105, 2024-04-26T18:34:43.933 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000818686 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 222, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 11, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-04-26T18:34:43.934 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 39, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 4, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 35, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 31, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 112, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 2, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 2, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 5, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 582, 2024-04-26T18:34:43.935 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 263, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 220, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 61, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 2, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 1, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 6, 2024-04-26T18:34:43.936 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4573, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 120763364, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 6, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:43.937 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 139, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 143, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 146, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 7, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 5, 2024-04-26T18:34:43.938 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 23 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 2, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 473, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 266, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 266, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 3, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 1, 2024-04-26T18:34:43.939 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 8, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 7, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-04-26T18:34:43.940 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 4, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 109, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:34:43.941 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 49, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 49, 2024-04-26T18:34:43.942 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-04-26T18:34:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-04-26T18:34:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 2, 2024-04-26T18:34:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-04-26T18:34:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-04-26T18:34:43.943 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 2, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 4267014, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-04-26T18:34:43.944 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 29, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.164627952, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005676825 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 36, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 99, 2024-04-26T18:34:43.945 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 63, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 28, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 239, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 211, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 40, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 164, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 124, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 2, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 23, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 21, 2024-04-26T18:34:43.946 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 55452, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 223608 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 283, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 7, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 260, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 29, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 374, 2024-04-26T18:34:43.947 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.008507800, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.004253900 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000388687, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000194343 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:43.948 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.949 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 19, 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.005741606, 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000302189 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-04-26T18:34:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2, 2024-04-26T18:34:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000197112, 2024-04-26T18:34:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000098556 2024-04-26T18:34:43.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:43.952 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.953 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-04-26T18:34:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-04-26T18:34:43.954 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.002485864, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002485864 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.955 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000202383, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000202383 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001470635, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000490211 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.956 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.957 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.001924709, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001924709 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.959 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-04-26T18:34:43.960 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.961 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 7, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.010191309, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001455901 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.962 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 2, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 5, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 3, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 2, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 18, 2024-04-26T18:34:43.963 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 9, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 591, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:43.964 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:43.965 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:43.966 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-04-26T18:34:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:43.967 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 296996, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 117, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-04-26T18:34:43.970 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 161876, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 941, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:43.971 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 106, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 87752, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 33, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 106, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 106, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.531987960, 2024-04-26T18:34:43.972 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005018754 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 106, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 145 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 106, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 1, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 105, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 0, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 10, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 30, 2024-04-26T18:34:43.973 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 18, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 19, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 20, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:43.974 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-04-26T18:34:43.975 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 102, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 14, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 11, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 11, 2024-04-26T18:34:43.976 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 39, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 0, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 8 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 113 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.977 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 7, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 7, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 7, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 7, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:43.978 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 6070, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 5376995, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 6070, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 6070, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 5376995, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:43.979 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 106, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 88667, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 106, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 88667, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.980 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 106, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 106, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 106, 2024-04-26T18:34:43.981 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 106, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 7, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 915, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 7, 2024-04-26T18:34:43.982 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 1, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 915, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55b0c2676500": { 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:43.983 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 49, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 72710, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 49, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 29, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 72710, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:43.984 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-26T18:34:44.675 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=2 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=473 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:34:44.676 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-26T18:34:44.676 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T18:34:44.676 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T18:34:44.912 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:34:45.174 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:44 smithi120 ceph-mon[32244]: pgmap v324: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:45.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:44 smithi037 ceph-mon[29171]: pgmap v324: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:45.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:44 smithi170 ceph-mon[32697]: pgmap v324: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 0, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 0, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 0, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.001062013, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T18:34:46.647 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000298261, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:46.648 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 177, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 188352, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.010919443, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.007301851, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001377614, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:46.649 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 174, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.015889219, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000091317 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 188352 2024-04-26T18:34:46.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 247, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 234, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 1598736, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 143232, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.048463649, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.014169760, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.037915117, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.005620380, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 234, 2024-04-26T18:34:46.651 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.012303039, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000052577 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 1598736, 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 143232 2024-04-26T18:34:46.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:46.653 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:46.654 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:46.655 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148721, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 30, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:46.656 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:46.657 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 245, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 1561397, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 245, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 245, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 1561397, 2024-04-26T18:34:46.658 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-26T18:34:46.659 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-26T18:34:47.205 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:46 smithi120 ceph-mon[32244]: pgmap v325: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:47.232 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T18:34:47.232 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T18:34:47.232 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-26T18:34:47.233 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-26T18:34:47.233 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T18:34:47.233 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T18:34:47.233 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T18:34:47.233 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-26T18:34:47.233 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T18:34:47.233 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T18:34:47.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:46 smithi037 ceph-mon[29171]: pgmap v325: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:47.261 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:46 smithi170 ceph-mon[32697]: pgmap v325: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:47.486 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:34:49.200 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:48 smithi170 ceph-mon[32697]: pgmap v326: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:49.227 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 329, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 329, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 265544, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 122106, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.045601730, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.014487068, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.020223276, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.012423848, 2024-04-26T18:34:49.228 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 328, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019518982, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000059509 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 179392, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 44192 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:49.229 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 516, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 565, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 228416, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 339140, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.037681599, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.013328635, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.009404246, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.012072656, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 564, 2024-04-26T18:34:49.230 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.025809245, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000045761 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 176576 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 3, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 5, 2024-04-26T18:34:49.231 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1342, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 961, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.001310318, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.000061901, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.001081346, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000085111, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001659406, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000553135 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:49.232 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.233 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1259, 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.035497139, 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028194 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:49.234 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-26T18:34:49.235 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 9, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 9, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 9, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-04-26T18:34:49.236 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-04-26T18:34:49.237 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:49.238 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-26T18:34:49.239 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-04-26T18:34:49.240 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-26T18:34:49.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 47, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 1, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 1, 2024-04-26T18:34:49.242 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 88092917, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 88187108, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 88186996, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 47 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 14, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 14, 2024-04-26T18:34:49.243 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 12, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 12, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 11, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 11, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 47744, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215392 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-26T18:34:49.244 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.246 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-26T18:34:49.247 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.248 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-26T18:34:49.249 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:48 smithi037 ceph-mon[29171]: pgmap v326: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-26T18:34:49.251 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.252 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.253 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.254 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.255 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 0, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 0, 2024-04-26T18:34:49.257 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1800, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:49.258 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:49.259 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:49.260 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 189698, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 61, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 172, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:49.261 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 86385, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 614, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 759, 2024-04-26T18:34:49.262 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 26, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 4, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 759, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 759, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.398990400, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000525679 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 759, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 760 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 759, 2024-04-26T18:34:49.263 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 759, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 488, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 268, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-04-26T18:34:49.264 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:49.265 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-26T18:34:49.266 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 102, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 8, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 5, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 5, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-04-26T18:34:49.267 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:34:49.268 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 844, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 412034, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 844, 2024-04-26T18:34:49.269 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 844, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 412034, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:49.270 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 759, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 94201, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 759, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 94201, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.271 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 759, 2024-04-26T18:34:49.272 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 759, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 759, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 759, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:49.273 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55b7b1ad6640": { 2024-04-26T18:34:49.274 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:49.275 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:49.276 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.276 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:49.276 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-26T18:34:49.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:48 smithi120 ceph-mon[32244]: pgmap v326: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:49.948 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T18:34:49.948 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T18:34:49.948 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-26T18:34:49.948 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-26T18:34:49.949 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-26T18:34:49.949 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:34:49.949 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:34:49.949 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-26T18:34:49.949 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T18:34:49.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T18:34:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:34:51.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:50 smithi037 ceph-mon[29171]: pgmap v327: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:51.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:50 smithi120 ceph-mon[32244]: pgmap v327: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:51.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:50 smithi170 ceph-mon[32697]: pgmap v327: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:51.901 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 543, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 521, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 5920278, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 230792, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 2, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.077163785, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.024792509, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.044650068, 2024-04-26T18:34:51.902 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.014358062, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 521, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.022567756, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000043316 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1658544, 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 144608 2024-04-26T18:34:51.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 626, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 799, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 28689826, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 663557, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.075586813, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.023708584, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.028204861, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.016450607, 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:51.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 796, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.037459428, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000047059 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 465120 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.905 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1497, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1499, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 25624315, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 476104, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.154888641, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.042382499, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.051662659, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.047949102, 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:51.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1497, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.062492586, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000041745 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-26T18:34:51.907 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4032, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.113345296, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000028111 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:51.908 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 14993, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 203, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 14790, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-04-26T18:34:51.910 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 14702, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-04-26T18:34:51.911 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:51.912 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-04-26T18:34:51.913 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-04-26T18:34:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 21523, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-04-26T18:34:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 31, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 3, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 62076828, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 62074740, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.917 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 21523 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 4963, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 10021, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 5058, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 223, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 255, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 32, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 14994, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 15012, 2024-04-26T18:34:51.918 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 18, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 80992, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215416 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-04-26T18:34:51.919 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.921 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.923 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.924 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.925 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.926 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-26T18:34:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 3, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 2985, 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 410389, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 5109, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 172, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 22256128, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 184963, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-26T18:34:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 2392, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 2392, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2392, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.540963019, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000644215 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2392, 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2393 2024-04-26T18:34:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 2392, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2392, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1499, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 891, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-26T18:34:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-26T18:34:51.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-26T18:34:51.941 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 102, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 14, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 9, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 9, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-26T18:34:51.942 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.943 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2662, 2024-04-26T18:34:51.944 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 59978076, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2662, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2662, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 59978076, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:51.945 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2392, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 57899529, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2392, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 57899529, 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:51.946 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2392, 2024-04-26T18:34:51.947 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2392, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2392, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2392, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:51.948 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5631633f46e0": { 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-26T18:34:51.950 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:51.951 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:34:52.712 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-26T18:34:52.713 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T18:34:52.713 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T18:34:52.952 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:34:53.213 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:52 smithi170 ceph-mon[32697]: pgmap v328: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:53.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:52 smithi037 ceph-mon[29171]: pgmap v328: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:53.253 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:52 smithi120 ceph-mon[32244]: pgmap v328: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 9150, 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 10837, 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 3344995, 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 30023848, 2024-04-26T18:34:54.712 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 10, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.638917673, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.274581223, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.281804666, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.097536734, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 10839, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.418763976, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000038634 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 13558, 2024-04-26T18:34:54.713 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002131172, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000157 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 2554, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 2560, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2242090, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 20518123, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-04-26T18:34:54.714 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.341006710, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.165461126, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.089104313, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.076998522, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2559, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.202815269, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000079255 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 55, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000009508, 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000172 2024-04-26T18:34:54.715 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1672000, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 160592 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 29740, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 50408, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 9536505, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 36542861, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 16, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 8, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.556511578, 2024-04-26T18:34:54.716 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.742301048, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.383659025, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.051878546, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 50402, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.352946352, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026843 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 66400, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.008162618, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000122 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.717 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 6304, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 470832 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2986, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.470745411, 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004176405 2024-04-26T18:34:54.718 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7890, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.512223830, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000318406 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 14931, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 14862, 2024-04-26T18:34:54.719 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14862, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 33.325278420, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002242314 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 69, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 175, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 20, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-26T18:34:54.720 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 9656, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 2, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 9654, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 9651, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 10, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 1, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 17, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 22009, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 19323, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 63, 2024-04-26T18:34:54.721 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 24, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 52, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 18, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1293, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4573, 2024-04-26T18:34:54.722 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 120763364, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 16, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 4579, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 9949, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 16, 2024-04-26T18:34:54.723 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 10289, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 5312, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 471, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 7413 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 371, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 23, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 15, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 37, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 456, 2024-04-26T18:34:54.724 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 55, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 5107, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 5052, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 4, 2024-04-26T18:34:54.725 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 9199, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 20, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 21523, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-26T18:34:54.726 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 21523, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 31, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 31, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 3, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 62076828, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-26T18:34:54.727 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1575, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.312141466, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004007708 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 4968, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 10024, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 5056, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 229, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 260, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 31, 2024-04-26T18:34:54.728 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 9657, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 15194, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 5537, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 1, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 9650, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 9649, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 117000, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223608 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 17610, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 951, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 14931, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 20, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 3283, 2024-04-26T18:34:54.729 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4074, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.887170318, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004636026 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 69, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.211148554, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003060123 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.730 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4962, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.486507122, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000501109 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-26T18:34:54.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.732 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-26T18:34:54.734 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.029590088, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001138080 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 112, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.054559602, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000487139 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 461, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.770005508, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001670293 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 72, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.153402695, 2024-04-26T18:34:54.736 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002130592 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-26T18:34:54.737 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 60, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.635873440, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.043931224 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.738 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 16, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.065513206, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004094575 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-26T18:34:54.739 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002734828, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000341853 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.740 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.006015118, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001203023 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4997, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.022757941, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001605514 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:34:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 3, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 8928, 2024-04-26T18:34:54.742 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 4464, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 543, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:54.743 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:54.744 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:54.745 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 898926, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 4802, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8232, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 172, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:54.746 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 18718308, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 190265, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 7096, 2024-04-26T18:34:54.747 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 61033141, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 7096, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7096, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 24.230414499, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003414658 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7096, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7373 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.748 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 7096, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 55, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 7041, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 55, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 1630, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 40, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 4, 2024-04-26T18:34:54.749 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 5095, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-26T18:34:54.750 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 16, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-26T18:34:54.751 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-26T18:34:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 102, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 14, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 465, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 38 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:34:54.753 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 9339 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 10, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 5054, 2024-04-26T18:34:54.754 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 5052, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 41440, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 11374588, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 41440, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 41440, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 11374588, 2024-04-26T18:34:54.755 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:54.756 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 7096, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 61686705, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 7096, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 61686705, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-26T18:34:54.757 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 7096, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 7096, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 7096, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 7096, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 5052, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 653564, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 5052, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 55, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 653564, 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:54.759 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55d93b326500": { 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 21523, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 57882524, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 21523, 2024-04-26T18:34:54.760 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1575, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 57882524, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:54.761 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-26T18:34:55.074 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:54 smithi170 ceph-mon[32697]: pgmap v329: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:55.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:54 smithi037 ceph-mon[29171]: pgmap v329: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:55.240 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:54 smithi120 ceph-mon[32244]: pgmap v329: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds.imported=18 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=371 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=23 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:34:55.377 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=20 2024-04-26T18:34:55.378 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T18:34:55.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T18:34:55.615 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 250, 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 237, 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1599552, 2024-04-26T18:34:57.234 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 145152, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 0, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.052291711, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.014460925, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.040777368, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.006041564, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 237, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.013221070, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000055785 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:57.235 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1599552, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 145152 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 2, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 180, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 320, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 188880, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-04-26T18:34:57.236 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.011226820, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.007499579, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.001487158, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 177, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.013653576, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000077138 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:34:57.237 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 188880 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 0, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 2, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 0, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 0, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 2, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.000319071, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.000264476, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:34:57.238 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 0, 2024-04-26T18:34:57.239 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:34:57.240 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-26T18:34:57.241 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 149735, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 30, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-26T18:34:57.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 928, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-26T18:34:57.243 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 248, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1561787, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 248, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-26T18:34:57.244 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 248, 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1561787, 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-26T18:34:57.245 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-26T18:34:57.245 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:57 smithi170 ceph-mon[32697]: pgmap v330: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:57.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:57 smithi037 ceph-mon[29171]: pgmap v330: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:57.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:57 smithi120 ceph-mon[32244]: pgmap v330: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T18:34:57.882 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T18:34:57.882 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T18:34:57.883 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-26T18:34:57.885 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-26T18:34:57.885 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T18:34:57.885 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T18:34:57.885 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T18:34:57.885 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-26T18:34:57.886 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T18:34:57.897 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T18:34:57.897 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:34:57.897 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:34:57.919 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T18:34:57.919 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:34:57.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T18:34:58.279 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:59.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:34:59 smithi037 ceph-mon[29171]: pgmap v331: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:59.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:34:59 smithi120 ceph-mon[32244]: pgmap v331: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:59.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:34:59 smithi170 ceph-mon[32697]: pgmap v331: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:34:59.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 -- 172.21.15.37:0/1336190510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01094e0 msgr2=0x7f26f01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 --2- 172.21.15.37:0/1336190510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01094e0 0x7f26f01098c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009990 tx=0x7f26e0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 -- 172.21.15.37:0/1336190510 shutdown_connections 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 --2- 172.21.15.37:0/1336190510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f0103e60 0x7f26f01102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 --2- 172.21.15.37:0/1336190510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01094e0 0x7f26f01098c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 --2- 172.21.15.37:0/1336190510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f01034a0 0x7f26f0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.963+0000 7f26f6934700 1 -- 172.21.15.37:0/1336190510 >> 172.21.15.37:0/1336190510 conn(0x7f26f007a590 msgr2=0x7f26f007a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.964+0000 7f26f6934700 1 -- 172.21.15.37:0/1336190510 shutdown_connections 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.964+0000 7f26f6934700 1 -- 172.21.15.37:0/1336190510 wait complete. 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.964+0000 7f26f6934700 1 Processor -- start 2024-04-26T18:34:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.964+0000 7f26f6934700 1 -- start start 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f0103e60 0x7f26f0075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f01094e0 0x7f26f00761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0113610 con 0x7f26f01034a0 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0113490 con 0x7f26f01094e0 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f6934700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0113790 con 0x7f26f0103e60 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36108/0 (socket says 172.21.15.37:36108) 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 -- 172.21.15.37:0/143646138 learned_addr learned my addr 172.21.15.37:0/143646138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:34:59.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26f4ed1700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f01094e0 0x7f26f00761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26ef7fe700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f0103e60 0x7f26f0075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 -- 172.21.15.37:0/143646138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f0103e60 msgr2=0x7f26f0075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f0103e60 0x7f26f0075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 -- 172.21.15.37:0/143646138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f01094e0 msgr2=0x7f26f00761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f01094e0 0x7f26f00761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 -- 172.21.15.37:0/143646138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f01ac4d0 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26effff700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000c00 tx=0x7f26e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.965+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0027070 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.966+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f01ac700 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.966+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e0003f20 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.966+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0022ca0 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.966+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f01acc40 con 0x7f26f01034a0 2024-04-26T18:34:59.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.967+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f010b950 con 0x7f26f01034a0 2024-04-26T18:34:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.968+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26e002a070 con 0x7f26f01034a0 2024-04-26T18:34:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.968+0000 7f26ed7fa700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 0x7f26d80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.968+0000 7f26ef7fe700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 0x7f26d80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.968+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f26e00d16b0 con 0x7f26f01034a0 2024-04-26T18:34:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.969+0000 7f26ef7fe700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 0x7f26d80a6270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26f0076ee0 tx=0x7f26e400b3c0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:34:59.973+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26e005eff0 con 0x7f26f01034a0 2024-04-26T18:35:00.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.205+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26f0076c50 con 0x7f26f01034a0 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.206+0000 7f26ed7fa700 1 -- 172.21.15.37:0/143646138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+938 (secure 0 0 0) 0x7f26e0013070 con 0x7f26f01034a0 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/813673186 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3120809922 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1405864097 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3994756262 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.206 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1399831496 2024-04-27T18:13:56.574428+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1233564213 2024-04-27T18:13:56.574428+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1233564213 2024-04-27T18:13:56.574428+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2586849717 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/199318008 2024-04-27T18:13:29.830032+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/813673186 2024-04-27T18:24:19.519750+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3928213229 2024-04-27T18:13:56.574428+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/53636636 2024-04-27T18:13:29.830032+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1726511030 2024-04-27T18:13:29.830032+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/776601124 2024-04-27T18:13:29.830032+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/155804720 2024-04-27T18:13:56.574428+0000 2024-04-26T18:35:00.207 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/199318008 2024-04-27T18:13:29.830032+0000 2024-04-26T18:35:00.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 msgr2=0x7f26d80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 0x7f26d80a6270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f26f0076ee0 tx=0x7f26e400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 msgr2=0x7f26f0075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26e0000c00 tx=0x7f26e00047b0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 shutdown_connections 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f26d80a3db0 0x7f26d80a6270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f01094e0 0x7f26f00761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26f01034a0 0x7f26f0075720 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 --2- 172.21.15.37:0/143646138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f0103e60 0x7f26f0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 >> 172.21.15.37:0/143646138 conn(0x7f26f007a590 msgr2=0x7f26f0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 shutdown_connections 2024-04-26T18:35:00.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:00.208+0000 7f26f6934700 1 -- 172.21.15.37:0/143646138 wait complete. 2024-04-26T18:35:00.210 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-04-26T18:35:00.831 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T18:35:00.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:35:00.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T18:35:00.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:35:00.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:35:00.991 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T18:35:01.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:35:01.021 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T18:35:01.082 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T18:35:01.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:35:01.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T18:35:01.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:35:01.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T18:35:01.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:01 smithi037 ceph-mon[29171]: pgmap v332: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:35:01.253 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:01 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/143646138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:35:01.254 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T18:35:01.266 INFO:tasks.cephadm:Teardown begin 2024-04-26T18:35:01.266 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T18:35:01.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:35:01.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:01 smithi120 ceph-mon[32244]: pgmap v332: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:35:01.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:01 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/143646138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:35:01.515 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:01 smithi170 ceph-mon[32697]: pgmap v332: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:35:01.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:01 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/143646138' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:35:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2467512341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 msgr2=0x7fd498110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2467512341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd498110290 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd49000b0a0 tx=0x7fd49000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2467512341 shutdown_connections 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2467512341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd498110290 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2467512341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4981094c0 0x7fd4981098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2467512341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd498103470 0x7fd4981038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.144+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2467512341 >> 172.21.15.37:0/2467512341 conn(0x7fd49807a580 msgr2=0x7fd49807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.145+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2467512341 shutdown_connections 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.145+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2467512341 wait complete. 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.145+0000 7fd49cfbc700 1 Processor -- start 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.145+0000 7fd49cfbc700 1 -- start start 2024-04-26T18:35:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498103470 0x7fd49810f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4981094c0 0x7fd4981a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981135c0 con 0x7fd498103e30 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd498113440 con 0x7fd498103470 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49cfbc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd498113740 con 0x7fd4981094c0 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd49659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498103470 0x7fd49810f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd496d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4981094c0 0x7fd4981a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36124/0 (socket says 172.21.15.37:36124) 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 -- 172.21.15.37:0/2584487754 learned_addr learned my addr 172.21.15.37:0/2584487754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 -- 172.21.15.37:0/2584487754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4981094c0 msgr2=0x7fd4981a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4981094c0 0x7fd4981a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 -- 172.21.15.37:0/2584487754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498103470 msgr2=0x7fd49810f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498103470 0x7fd49810f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.146+0000 7fd48ffff700 1 -- 172.21.15.37:0/2584487754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4981a5010 con 0x7fd498103e30 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.147+0000 7fd48ffff700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d950 tx=0x7fd48000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.147+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480016070 con 0x7fd498103e30 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.147+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981a5300 con 0x7fd498103e30 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.147+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48000fd40 con 0x7fd498103e30 2024-04-26T18:35:03.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.147+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48000e3b0 con 0x7fd498103e30 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.148+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981acb80 con 0x7fd498103e30 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.149+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49806a110 con 0x7fd498103e30 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.149+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd480019070 con 0x7fd498103e30 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.150+0000 7fd48f7fe700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 0x7fd4840a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.150+0000 7fd49659c700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 0x7fd4840a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:03.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.150+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fd4800cfd20 con 0x7fd498103e30 2024-04-26T18:35:03.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.151+0000 7fd49659c700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 0x7fd4840a6270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd48800abd0 tx=0x7fd48800a380 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:03.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.155+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd48009c380 con 0x7fd498103e30 2024-04-26T18:35:03.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.414+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4981acdb0 con 0x7fd498103e30 2024-04-26T18:35:03.415 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:03 smithi037 ceph-mon[29171]: pgmap v333: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:35:03.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.415+0000 7fd48f7fe700 1 -- 172.21.15.37:0/2584487754 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11886 (secure 0 0 0) 0x7fd48009bc80 con 0x7fd498103e30 2024-04-26T18:35:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:35:03.416 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:30:45.074860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25618":{"gid":25618,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/1357982747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1357982747},{"type":"v1","addr":"172.21.15.170:6837","nonce":1357982747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:35:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 msgr2=0x7fd4840a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 0x7fd4840a6270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd48800abd0 tx=0x7fd48800a380 comp rx=0 tx=0).stop 2024-04-26T18:35:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 msgr2=0x7fd49810faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:03.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d950 tx=0x7fd48000dc60 comp rx=0 tx=0).stop 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 shutdown_connections 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4981094c0 0x7fd4981a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7fd4840a3db0 0x7fd4840a6270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd498103470 0x7fd49810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 --2- 172.21.15.37:0/2584487754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd498103e30 0x7fd49810faa0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 >> 172.21.15.37:0/2584487754 conn(0x7fd49807a580 msgr2=0x7fd498101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 shutdown_connections 2024-04-26T18:35:03.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:03.417+0000 7fd49cfbc700 1 -- 172.21.15.37:0/2584487754 wait complete. 2024-04-26T18:35:03.419 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-26T18:35:03.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:03 smithi120 ceph-mon[32244]: pgmap v333: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:35:03.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:03 smithi170 ceph-mon[32697]: pgmap v333: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-26T18:35:04.042 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:35:04.294 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:04.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:04 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2584487754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:04.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:04 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2584487754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:04.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:04 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2584487754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:05.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:05 smithi037 ceph-mon[29171]: pgmap v334: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:35:05.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:05 smithi120 ceph-mon[32244]: pgmap v334: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:35:05.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:05 smithi170 ceph-mon[32697]: pgmap v334: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T18:35:05.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/1654844607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb806bbd0 msgr2=0x7f4bb806c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/1654844607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb806bbd0 0x7f4bb806c050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009990 tx=0x7f4bac009ca0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/1654844607 shutdown_connections 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/1654844607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 0x7f4bb8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/1654844607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb81094d0 0x7f4bb81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/1654844607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb806bbd0 0x7f4bb806c050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/1654844607 >> 172.21.15.37:0/1654844607 conn(0x7f4bb80793c0 msgr2=0x7f4bb80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/1654844607 shutdown_connections 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.952+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/1654844607 wait complete. 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 Processor -- start 2024-04-26T18:35:05.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 -- start start 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb806bbd0 0x7f4bb819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 0x7f4bb819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8113610 con 0x7f4bb81094d0 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8113490 con 0x7f4bb806bbd0 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbe3b3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8113790 con 0x7f4bb806c590 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37480/0 (socket says 172.21.15.37:37480) 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 -- 172.21.15.37:0/4077845783 learned_addr learned my addr 172.21.15.37:0/4077845783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bb7fff700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb806bbd0 0x7f4bb819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:05.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bb77fe700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 0x7f4bb819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 -- 172.21.15.37:0/4077845783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 msgr2=0x7f4bb819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 0x7f4bb819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 -- 172.21.15.37:0/4077845783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb806bbd0 msgr2=0x7f4bb819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb806bbd0 0x7f4bb819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.953+0000 7f4bbc950700 1 -- 172.21.15.37:0/4077845783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb819a0d0 con 0x7f4bb81094d0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bbc950700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007ee0 tx=0x7f4bb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000dbd0 con 0x7f4bb81094d0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb000dd70 con 0x7f4bb81094d0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb819a3c0 con 0x7f4bb81094d0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0005990 con 0x7f4bb81094d0 2024-04-26T18:35:05.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.954+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a6680 con 0x7f4bb81094d0 2024-04-26T18:35:05.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.955+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb810b9b0 con 0x7f4bb81094d0 2024-04-26T18:35:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.956+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb001a020 con 0x7f4bb81094d0 2024-04-26T18:35:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.957+0000 7f4bb57fa700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 0x7f4ba00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.957+0000 7f4bb7fff700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 0x7f4ba00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.957+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f4bb00d0700 con 0x7f4bb81094d0 2024-04-26T18:35:05.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.957+0000 7f4bb7fff700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 0x7f4ba00a6270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bb819aab0 tx=0x7f4ba800b410 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:05.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:05.961+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bb0010070 con 0x7f4bb81094d0 2024-04-26T18:35:06.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.277+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bb819aed0 con 0x7f4bb81094d0 2024-04-26T18:35:06.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.280+0000 7f4bb57fa700 1 -- 172.21.15.37:0/4077845783 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11886 (secure 0 0 0) 0x7f4bb819aed0 con 0x7f4bb81094d0 2024-04-26T18:35:06.282 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:35:06.282 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1294150322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1294150322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1294150322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1073340191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1073340191},{"type":"v1","addr":"172.21.15.37:6837","nonce":1073340191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1645821272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1645821272},{"type":"v1","addr":"172.21.15.37:6839","nonce":1645821272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/3026662936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":3026662936},{"type":"v1","addr":"172.21.15.37:6841","nonce":3026662936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/1924785212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":1924785212},{"type":"v1","addr":"172.21.15.170:6833","nonce":1924785212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/1778061281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1778061281},{"type":"v1","addr":"172.21.15.120:6833","nonce":1778061281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:21:10.996859+0000","modified":"2024-04-26T18:30:45.074860+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24464,"mds_1":24449,"mds_2":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/707687219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":707687219},{"type":"v1","addr":"172.21.15.170:6835","nonce":707687219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3605086601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3605086601},{"type":"v1","addr":"172.21.15.120:6835","nonce":3605086601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/1569501885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":1569501885},{"type":"v1","addr":"172.21.15.170:6839","nonce":1569501885}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.120:6837/1266100832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1266100832},{"type":"v1","addr":"172.21.15.120:6837","nonce":1266100832}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.120:6839/1246417570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":1246417570},{"type":"v1","addr":"172.21.15.120:6839","nonce":1246417570}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25618":{"gid":25618,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/1357982747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1357982747},{"type":"v1","addr":"172.21.15.170:6837","nonce":1357982747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24464,"qdb_cluster":[24464,24442,24449]},"id":1}]} 2024-04-26T18:35:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.284+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 msgr2=0x7f4ba00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.284+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 0x7f4ba00a6270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bb819aab0 tx=0x7f4ba800b410 comp rx=0 tx=0).stop 2024-04-26T18:35:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.284+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 msgr2=0x7f4bb8199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.284+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007ee0 tx=0x7f4bb000d3b0 comp rx=0 tx=0).stop 2024-04-26T18:35:06.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 shutdown_connections 2024-04-26T18:35:06.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bb81094d0 0x7f4bb8199970 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:06.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f4ba00a3db0 0x7f4ba00a6270 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bb819aab0 tx=0x7f4ba800b410 comp rx=0 tx=0).stop 2024-04-26T18:35:06.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4bb806bbd0 0x7f4bb819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:06.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 --2- 172.21.15.37:0/4077845783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4bb806c590 0x7f4bb819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:06.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 >> 172.21.15.37:0/4077845783 conn(0x7f4bb80793c0 msgr2=0x7f4bb81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:06.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.285+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 shutdown_connections 2024-04-26T18:35:06.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:06.287+0000 7f4bbe3b3700 1 -- 172.21.15.37:0/4077845783 wait complete. 2024-04-26T18:35:06.289 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-26T18:35:07.010 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:35:07.382 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:07.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:07 smithi037 ceph-mon[29171]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-26T18:35:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:07 smithi037 ceph-mon[29171]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:35:07.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:07 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4077845783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:07.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:07 smithi120 ceph-mon[32244]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-26T18:35:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:07 smithi120 ceph-mon[32244]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:35:07.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:07 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4077845783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:07 smithi170 ceph-mon[32697]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-04-26T18:35:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:07 smithi170 ceph-mon[32697]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:35:07.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:07 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4077845783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 -- 172.21.15.37:0/4012546053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b006bbd0 msgr2=0x7f44b006c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/4012546053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b006bbd0 0x7f44b006c050 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009990 tx=0x7f44a4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 -- 172.21.15.37:0/4012546053 shutdown_connections 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/4012546053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006c590 0x7f44b0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/4012546053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b01094d0 0x7f44b01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/4012546053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b006bbd0 0x7f44b006c050 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 -- 172.21.15.37:0/4012546053 >> 172.21.15.37:0/4012546053 conn(0x7f44b00793c0 msgr2=0x7f44b00797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 -- 172.21.15.37:0/4012546053 shutdown_connections 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.117+0000 7f44b4bb2700 1 -- 172.21.15.37:0/4012546053 wait complete. 2024-04-26T18:35:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 Processor -- start 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 -- start start 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006bbd0 0x7f44b019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b006c590 0x7f44b019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01135e0 con 0x7f44b01094d0 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0113460 con 0x7f44b006bbd0 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44b4bb2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0113760 con 0x7f44b006c590 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37510/0 (socket says 172.21.15.37:37510) 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 -- 172.21.15.37:0/283918838 learned_addr learned my addr 172.21.15.37:0/283918838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44ae59c700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006bbd0 0x7f44b019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44add9b700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b006c590 0x7f44b019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 -- 172.21.15.37:0/283918838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b006c590 msgr2=0x7f44b019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b006c590 0x7f44b019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 -- 172.21.15.37:0/283918838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006bbd0 msgr2=0x7f44b019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006bbd0 0x7f44b019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 -- 172.21.15.37:0/283918838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b019a0a0 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.118+0000 7f44aed9d700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f44a800c970 tx=0x7f44a800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.119+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a800ce60 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.119+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b019a390 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.119+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8004830 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.119+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a80054c0 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.119+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b01a6650 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.120+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b006a110 con 0x7f44b01094d0 2024-04-26T18:35:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.121+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44a80049e0 con 0x7f44b01094d0 2024-04-26T18:35:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.122+0000 7f449f7fe700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 0x7f44980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.122+0000 7f44ae59c700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 0x7f44980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.122+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8623+0+0 (secure 0 0 0) 0x7f44a80d0710 con 0x7f44b01094d0 2024-04-26T18:35:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.122+0000 7f44ae59c700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 0x7f44980a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009550 tx=0x7f44a00091b0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.126+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44a8014030 con 0x7f44b01094d0 2024-04-26T18:35:09.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.340+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44b019aea0 con 0x7f44b01094d0 2024-04-26T18:35:09.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.342+0000 7f449f7fe700 1 -- 172.21.15.37:0/283918838 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23261 (secure 0 0 0) 0x7f44b019aea0 con 0x7f44b01094d0 2024-04-26T18:35:09.343 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T18:35:09.343 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"fsid":"755ef512-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:12:54.869827+0000","modified":"2024-04-26T18:35:06.164845+0000","last_up_change":"2024-04-26T18:20:04.818499+0000","last_in_change":"2024-04-26T18:19:44.660369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:16:20.665733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:21:04.549606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:21:07.522605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c564014-c9c6-4050-8c9e-ab1b1825a7cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6803","nonce":580247654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6805","nonce":580247654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6809","nonce":580247654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":580247654},{"type":"v1","addr":"172.21.15.37:6807","nonce":580247654}]},"public_addr":"172.21.15.37:6803/580247654","cluster_addr":"172.21.15.37:6805/580247654","heartbeat_back_addr":"172.21.15.37:6809/580247654","heartbeat_front_addr":"172.21.15.37:6807/580247654","state":["exists","up"]},{"osd":1,"uuid":"44d22495-dc0e-4795-9b4c-f9f1d6c4546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6801","nonce":634623221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6803","nonce":634623221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6807","nonce":634623221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":634623221},{"type":"v1","addr":"172.21.15.120:6805","nonce":634623221}]},"public_addr":"172.21.15.120:6801/634623221","cluster_addr":"172.21.15.120:6803/634623221","heartbeat_back_addr":"172.21.15.120:6807/634623221","heartbeat_front_addr":"172.21.15.120:6805/634623221","state":["exists","up"]},{"osd":2,"uuid":"5b1c3764-35d5-4229-b362-722dbbe07c44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6801","nonce":1287805124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6803","nonce":1287805124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6807","nonce":1287805124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":1287805124},{"type":"v1","addr":"172.21.15.170:6805","nonce":1287805124}]},"public_addr":"172.21.15.170:6801/1287805124","cluster_addr":"172.21.15.170:6803/1287805124","heartbeat_back_addr":"172.21.15.170:6807/1287805124","heartbeat_front_addr":"172.21.15.170:6805/1287805124","state":["exists","up"]},{"osd":3,"uuid":"26ddcdf7-9137-4384-9228-8537b2009989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6811","nonce":3804676062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6813","nonce":3804676062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6817","nonce":3804676062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3804676062},{"type":"v1","addr":"172.21.15.37:6815","nonce":3804676062}]},"public_addr":"172.21.15.37:6811/3804676062","cluster_addr":"172.21.15.37:6813/3804676062","heartbeat_back_addr":"172.21.15.37:6817/3804676062","heartbeat_front_addr":"172.21.15.37:6815/3804676062","state":["exists","up"]},{"osd":4,"uuid":"633a4362-a88a-4725-b10e-6c893bacd0ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6809","nonce":4165015053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6811","nonce":4165015053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6815","nonce":4165015053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":4165015053},{"type":"v1","addr":"172.21.15.120:6813","nonce":4165015053}]},"public_addr":"172.21.15.120:6809/4165015053","cluster_addr":"172.21.15.120:6811/4165015053","heartbeat_back_addr":"172.21.15.120:6815/4165015053","heartbeat_front_addr":"172.21.15.120:6813/4165015053","state":["exists","up"]},{"osd":5,"uuid":"493dab6b-f382-47e3-9a56-480a1328eda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6809","nonce":3652828620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6811","nonce":3652828620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6815","nonce":3652828620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3652828620},{"type":"v1","addr":"172.21.15.170:6813","nonce":3652828620}]},"public_addr":"172.21.15.170:6809/3652828620","cluster_addr":"172.21.15.170:6811/3652828620","heartbeat_back_addr":"172.21.15.170:6815/3652828620","heartbeat_front_addr":"172.21.15.170:6813/3652828620","state":["exists","up"]},{"osd":6,"uuid":"630f5f5f-942b-4a8c-be69-8b1f55491a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6819","nonce":1123986561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6821","nonce":1123986561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6825","nonce":1123986561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":1123986561},{"type":"v1","addr":"172.21.15.37:6823","nonce":1123986561}]},"public_addr":"172.21.15.37:6819/1123986561","cluster_addr":"172.21.15.37:6821/1123986561","heartbeat_back_addr":"172.21.15.37:6825/1123986561","heartbeat_front_addr":"172.21.15.37:6823/1123986561","state":["exists","up"]},{"osd":7,"uuid":"78b9cc68-5517-46de-858e-342ba834c2f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6817","nonce":152962141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6819","nonce":152962141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6823","nonce":152962141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":152962141},{"type":"v1","addr":"172.21.15.120:6821","nonce":152962141}]},"public_addr":"172.21.15.120:6817/152962141","cluster_addr":"172.21.15.120:6819/152962141","heartbeat_back_addr":"172.21.15.120:6823/152962141","heartbeat_front_addr":"172.21.15.120:6821/152962141","state":["exists","up"]},{"osd":8,"uuid":"8988738f-786a-4a7e-90da-7bab9c98319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6817","nonce":428251464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6819","nonce":428251464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6823","nonce":428251464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":428251464},{"type":"v1","addr":"172.21.15.170:6821","nonce":428251464}]},"public_addr":"172.21.15.170:6817/428251464","cluster_addr":"172.21.15.170:6819/428251464","heartbeat_back_addr":"172.21.15.170:6823/428251464","heartbeat_front_addr":"172.21.15.170:6821/428251464","state":["exists","up"]},{"osd":9,"uuid":"c35fc73f-65b6-4db3-8180-ee78649aab29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6827","nonce":4227134134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6829","nonce":4227134134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6833","nonce":4227134134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":4227134134},{"type":"v1","addr":"172.21.15.37:6831","nonce":4227134134}]},"public_addr":"172.21.15.37:6827/4227134134","cluster_addr":"172.21.15.37:6829/4227134134","heartbeat_back_addr":"172.21.15.37:6833/4227134134","heartbeat_front_addr":"172.21.15.37:6831/4227134134","state":["exists","up"]},{"osd":10,"uuid":"416522bb-db89-4bfd-83ca-082a3abd1dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6825","nonce":2391866955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6827","nonce":2391866955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6831","nonce":2391866955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2391866955},{"type":"v1","addr":"172.21.15.120:6829","nonce":2391866955}]},"public_addr":"172.21.15.120:6825/2391866955","cluster_addr":"172.21.15.120:6827/2391866955","heartbeat_back_addr":"172.21.15.120:6831/2391866955","heartbeat_front_addr":"172.21.15.120:6829/2391866955","state":["exists","up"]},{"osd":11,"uuid":"54e2bea7-5941-4f6b-ab50-2d7ab2ea1b23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6825","nonce":49311375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6827","nonce":49311375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6831","nonce":49311375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":49311375},{"type":"v1","addr":"172.21.15.170:6829","nonce":49311375}]},"public_addr":"172.21.15.170:6825/49311375","cluster_addr":"172.21.15.170:6827/49311375","heartbeat_back_addr":"172.21.15.170:6831/49311375","heartbeat_front_addr":"172.21.15.170:6829/49311375","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:30.283620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:15:54.419736+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:18.278794+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:16:43.141212+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:07.469563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:31.509643+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:17:57.578833+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:22.380499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:46.349525+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:12.269107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:37.509066+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:02.439032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3120809922":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1405864097":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3994756262":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/1399831496":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6801/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/1233564213":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/2586849717":"2024-04-27T18:24:19.519750+0000","172.21.15.37:6801/199318008":"2024-04-27T18:13:29.830032+0000","172.21.15.37:6800/813673186":"2024-04-27T18:24:19.519750+0000","172.21.15.37:0/3928213229":"2024-04-27T18:13:56.574428+0000","172.21.15.37:0/53636636":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/1726511030":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/776601124":"2024-04-27T18:13:29.830032+0000","172.21.15.37:0/155804720":"2024-04-27T18:13:56.574428+0000","172.21.15.37:6800/199318008":"2024-04-27T18:13:29.830032+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:35:09.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 msgr2=0x7f44980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:09.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 0x7f44980a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009550 tx=0x7f44a00091b0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 msgr2=0x7f44b0199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f44a800c970 tx=0x7f44a800cc80 comp rx=0 tx=0).stop 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 shutdown_connections 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b01094d0 0x7f44b0199940 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f44980a3db0 0x7f44980a6270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44b006bbd0 0x7f44b019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 --2- 172.21.15.37:0/283918838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b006c590 0x7f44b019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 >> 172.21.15.37:0/283918838 conn(0x7f44b00793c0 msgr2=0x7f44b01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 shutdown_connections 2024-04-26T18:35:09.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:09.344+0000 7f44b4bb2700 1 -- 172.21.15.37:0/283918838 wait complete. 2024-04-26T18:35:09.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:09 smithi037 ceph-mon[29171]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:09.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:09 smithi120 ceph-mon[32244]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:09.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:09 smithi170 ceph-mon[32697]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:10.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-26T18:35:10.270 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:10.283 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:10 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/283918838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:35:10.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:10 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/283918838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:35:10.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:10 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/283918838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:35:11.286 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:11 smithi037 ceph-mon[29171]: pgmap v338: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:11.476 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:11 smithi170 ceph-mon[32697]: pgmap v338: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:11.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:11 smithi120 ceph-mon[32244]: pgmap v338: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 -- 172.21.15.37:0/1392154394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194107640 msgr2=0x7f119410f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 --2- 172.21.15.37:0/1392154394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194107640 0x7f119410f210 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f118c00b0d0 tx=0x7f118c00b3e0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 -- 172.21.15.37:0/1392154394 shutdown_connections 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 --2- 172.21.15.37:0/1392154394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194107640 0x7f119410f210 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 --2- 172.21.15.37:0/1392154394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 0x7f1194102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 --2- 172.21.15.37:0/1392154394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194102f60 0x7f1194106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 -- 172.21.15.37:0/1392154394 >> 172.21.15.37:0/1392154394 conn(0x7f11940792d0 msgr2=0x7f11940796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 -- 172.21.15.37:0/1392154394 shutdown_connections 2024-04-26T18:35:11.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.900+0000 7f1199e07700 1 -- 172.21.15.37:0/1392154394 wait complete. 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 Processor -- start 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 -- start start 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 0x7f119419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194107640 0x7f11941a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1194112530 con 0x7f1194102f60 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11941123b0 con 0x7f11941025b0 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.901+0000 7f1199e07700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11941126b0 con 0x7f1194107640 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37532/0 (socket says 172.21.15.37:37532) 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 -- 172.21.15.37:0/1863045180 learned_addr learned my addr 172.21.15.37:0/1863045180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1193fff700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194107640 0x7f11941a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f11937fe700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 0x7f119419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:11.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 -- 172.21.15.37:0/1863045180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194107640 msgr2=0x7f11941a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194107640 0x7f11941a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 -- 172.21.15.37:0/1863045180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 msgr2=0x7f119419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 0x7f119419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 -- 172.21.15.37:0/1863045180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11941a5e90 con 0x7f1194102f60 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1192ffd700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f118800d950 tx=0x7f118800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188016070 con 0x7f1194102f60 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11941a60c0 con 0x7f1194102f60 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188004830 con 0x7f1194102f60 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188005440 con 0x7f1194102f60 2024-04-26T18:35:11.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.902+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11941a65d0 con 0x7f1194102f60 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.903+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119406a110 con 0x7f1194102f60 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.905+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11880055e0 con 0x7f1194102f60 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.905+0000 7f1190ff9700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 0x7f117c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.905+0000 7f11937fe700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 0x7f117c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.905+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8623+0+0 (secure 0 0 0) 0x7f11880d0a80 con 0x7f1194102f60 2024-04-26T18:35:11.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.906+0000 7f11937fe700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 0x7f117c0a6270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1184005fd0 tx=0x7f1184005e30 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:11.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:11.909+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f118801f020 con 0x7f1194102f60 2024-04-26T18:35:12.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.162+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f119402e650 con 0x7f1194102f60 2024-04-26T18:35:12.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.165+0000 7f1190ff9700 1 -- 172.21.15.37:0/1863045180 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f118809d0c0 con 0x7f1194102f60 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 msgr2=0x7f117c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 0x7f117c0a6270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1184005fd0 tx=0x7f1184005e30 comp rx=0 tx=0).stop 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 msgr2=0x7f119419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f118800d950 tx=0x7f118800dc60 comp rx=0 tx=0).stop 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 shutdown_connections 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1194107640 0x7f11941a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f117c0a3db0 0x7f117c0a6270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11941025b0 0x7f119419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 --2- 172.21.15.37:0/1863045180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1194102f60 0x7f119419ae10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.167+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 >> 172.21.15.37:0/1863045180 conn(0x7f11940792d0 msgr2=0x7f1194100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.168+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 shutdown_connections 2024-04-26T18:35:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:12.168+0000 7f1199e07700 1 -- 172.21.15.37:0/1863045180 wait complete. 2024-04-26T18:35:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T18:35:12.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a[29167]: 2024-04-26T18:35:12.163+0000 7f5260247700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:35:12.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:12 smithi037 ceph-mon[29171]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T18:35:12.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mon[32244]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mds[62602]: -- 172.21.15.120:0/937490051 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b53077a40 con 0x557b5304ac00 2024-04-26T18:35:12.503 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: starting mds.h at 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mds[62913]: -- 172.21.15.120:0/1441048198 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611be6d3a40 con 0x5611be6a6c00 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: starting mds.b at 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e[62277]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e[62277]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-mds[62281]: -- 172.21.15.120:0/1965730397 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635b359fa40 con 0x5635b3573000 2024-04-26T18:35:12.504 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:35:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e[62277]: starting mds.e at 2024-04-26T18:35:12.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/1863045180' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mon[32697]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mds[63003]: -- 172.21.15.170:0/1208448142 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563caf98da40 con 0x563caf960800 2024-04-26T18:35:12.576 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: starting mds.c at 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: starting mds.f at 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: ignoring --setuser ceph since I am not root 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: ignoring --setgroup ceph since I am not root 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-mds[63311]: -- 172.21.15.170:0/2717221475 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608f52b1a40 con 0x5608f5284800 2024-04-26T18:35:12.577 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:35:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: starting mds.i at 2024-04-26T18:35:12.787 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T18:35:13.037 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:13.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:13 smithi037 ceph-mon[29171]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T18:35:13.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:13 smithi120 ceph-mon[32244]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T18:35:13.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:13 smithi170 ceph-mon[32697]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6835/3940608816,v1:172.21.15.170:6837/3940608816] up:boot 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6834/3769833822,v1:172.21.15.170:6836/3769833822] up:boot 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.170:6838/1914377232,v1:172.21.15.170:6839/1914377232] up:boot 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6834/1421842552,v1:172.21.15.120:6836/1421842552] up:boot 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6835/3086259124,v1:172.21.15.120:6837/3086259124] up:boot 2024-04-26T18:35:14.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: mds.? [v2:172.21.15.120:6838/1089051574,v1:172.21.15.120:6839/1089051574] up:boot 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:35:14.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:14 smithi037 ceph-mon[29171]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6835/3940608816,v1:172.21.15.170:6837/3940608816] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6834/3769833822,v1:172.21.15.170:6836/3769833822] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.170:6838/1914377232,v1:172.21.15.170:6839/1914377232] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6834/1421842552,v1:172.21.15.120:6836/1421842552] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6835/3086259124,v1:172.21.15.120:6837/3086259124] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: mds.? [v2:172.21.15.120:6838/1089051574,v1:172.21.15.120:6839/1089051574] up:boot 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:35:14.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:35:14.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:35:14.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:35:14.504 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:14 smithi120 ceph-mon[32244]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:35:14.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6835/3940608816,v1:172.21.15.170:6837/3940608816] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6834/3769833822,v1:172.21.15.170:6836/3769833822] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.170:6838/1914377232,v1:172.21.15.170:6839/1914377232] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6834/1421842552,v1:172.21.15.120:6836/1421842552] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6835/3086259124,v1:172.21.15.120:6837/3086259124] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: mds.? [v2:172.21.15.120:6838/1089051574,v1:172.21.15.120:6839/1089051574] up:boot 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:35:14.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:14 smithi170 ceph-mon[32697]: from='mgr.24820 172.21.15.37:0/3864738080' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:35:14.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 -- 172.21.15.37:0/3130941083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102ce0 msgr2=0x7f337c10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 --2- 172.21.15.37:0/3130941083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102ce0 0x7f337c10f1d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f337400b0a0 tx=0x7f337400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 -- 172.21.15.37:0/3130941083 shutdown_connections 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 --2- 172.21.15.37:0/3130941083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102ce0 0x7f337c10f1d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 --2- 172.21.15.37:0/3130941083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c108390 0x7f337c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 --2- 172.21.15.37:0/3130941083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c102320 0x7f337c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 -- 172.21.15.37:0/3130941083 >> 172.21.15.37:0/3130941083 conn(0x7f337c0792d0 msgr2=0x7f337c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:14.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 -- 172.21.15.37:0/3130941083 shutdown_connections 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.874+0000 7f3382515700 1 -- 172.21.15.37:0/3130941083 wait complete. 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 Processor -- start 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 -- start start 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c102ce0 0x7f337c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c108390 0x7f337c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112500 con 0x7f337c102320 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112380 con 0x7f337c102ce0 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3382515700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112680 con 0x7f337c108390 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f337bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:14.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f337bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44998/0 (socket says 172.21.15.37:44998) 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f337bfff700 1 -- 172.21.15.37:0/3659733302 learned_addr learned my addr 172.21.15.37:0/3659733302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.875+0000 7f3380ab2700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c108390 0x7f337c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337b7fe700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c102ce0 0x7f337c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 -- 172.21.15.37:0/3659733302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c108390 msgr2=0x7f337c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c108390 0x7f337c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 -- 172.21.15.37:0/3659733302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c102ce0 msgr2=0x7f337c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c102ce0 0x7f337c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 -- 172.21.15.37:0/3659733302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c19a090 con 0x7f337c102320 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f337bfff700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d950 tx=0x7f336c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c016070 con 0x7f337c102320 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c009df0 con 0x7f337c102320 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c00f3c0 con 0x7f337c102320 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c19a380 con 0x7f337c102320 2024-04-26T18:35:14.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.876+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a6640 con 0x7f337c102320 2024-04-26T18:35:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.877+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c06a110 con 0x7f337c102320 2024-04-26T18:35:14.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.878+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f336c019070 con 0x7f337c102320 2024-04-26T18:35:14.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.879+0000 7f33797fa700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 0x7f33640a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:14.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.879+0000 7f337b7fe700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 0x7f33640a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:14.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.879+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8837+0+0 (secure 0 0 0) 0x7f336c0d0430 con 0x7f337c102320 2024-04-26T18:35:14.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.880+0000 7f337b7fe700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 0x7f33640a62f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3370005950 tx=0x7f33700058e0 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:14.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:14.883+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f336c09c990 con 0x7f337c102320 2024-04-26T18:35:15.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.140+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f337c1a6840 con 0x7f337c102320 2024-04-26T18:35:15.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.202+0000 7f33797fa700 1 -- 172.21.15.37:0/3659733302 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7f336c09c290 con 0x7f337c102320 2024-04-26T18:35:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 msgr2=0x7f33640a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 0x7f33640a62f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3370005950 tx=0x7f33700058e0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 msgr2=0x7f337c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d950 tx=0x7f336c00dc60 comp rx=0 tx=0).stop 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 shutdown_connections 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f337c108390 0x7f337c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f337c102320 0x7f337c19f770 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f33640a3e30 0x7f33640a62f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 --2- 172.21.15.37:0/3659733302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c102ce0 0x7f337c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 >> 172.21.15.37:0/3659733302 conn(0x7f337c0792d0 msgr2=0x7f337c1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 shutdown_connections 2024-04-26T18:35:15.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:15.204+0000 7f3382515700 1 -- 172.21.15.37:0/3659733302 wait complete. 2024-04-26T18:35:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:15 smithi037 ceph-mon[29171]: pgmap v341: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-26T18:35:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:15 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:35:15.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:15 smithi037 ceph-mon[29171]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:35:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:15 smithi120 ceph-mon[32244]: pgmap v341: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-26T18:35:15.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:15 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:35:15.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:15 smithi120 ceph-mon[32244]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:35:15.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:15 smithi170 ceph-mon[32697]: pgmap v341: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-26T18:35:15.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:15 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:35:15.577 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:15 smithi170 ceph-mon[32697]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:35:15.872 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T18:35:16.124 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:16.463 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:16 smithi037 ceph-mon[29171]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:16.464 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:16 smithi037 ceph-mon[29171]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:35:16.464 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:16 smithi037 ceph-mon[29171]: Cluster is now healthy 2024-04-26T18:35:16.464 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:16 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:35:16.464 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:16 smithi037 ceph-mon[29171]: fsmap 12 up:standby 2024-04-26T18:35:16.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:16 smithi120 ceph-mon[32244]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:16 smithi120 ceph-mon[32244]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:35:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:16 smithi120 ceph-mon[32244]: Cluster is now healthy 2024-04-26T18:35:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:16 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:35:16.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:16 smithi120 ceph-mon[32244]: fsmap 12 up:standby 2024-04-26T18:35:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:16 smithi170 ceph-mon[32697]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:16 smithi170 ceph-mon[32697]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:35:16.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:16 smithi170 ceph-mon[32697]: Cluster is now healthy 2024-04-26T18:35:16.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:16 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/3659733302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:35:16.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:16 smithi170 ceph-mon[32697]: fsmap 12 up:standby 2024-04-26T18:35:17.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:17 smithi037 ceph-mon[29171]: pgmap v343: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:17.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:17 smithi120 ceph-mon[32244]: pgmap v343: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:17.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:17 smithi170 ceph-mon[32697]: pgmap v343: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:17.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 -- 172.21.15.37:0/593500878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b201034c0 msgr2=0x7f2b20103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 --2- 172.21.15.37:0/593500878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b201034c0 0x7f2b20103940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2b140099c0 tx=0x7f2b14009cd0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 -- 172.21.15.37:0/593500878 shutdown_connections 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 --2- 172.21.15.37:0/593500878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 0x7f2b20110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 --2- 172.21.15.37:0/593500878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b20109560 0x7f2b20109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 --2- 172.21.15.37:0/593500878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b201034c0 0x7f2b20103940 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 -- 172.21.15.37:0/593500878 >> 172.21.15.37:0/593500878 conn(0x7f2b2007a5d0 msgr2=0x7f2b2007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 -- 172.21.15.37:0/593500878 shutdown_connections 2024-04-26T18:35:17.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.598+0000 7f2b2844b700 1 -- 172.21.15.37:0/593500878 wait complete. 2024-04-26T18:35:17.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 Processor -- start 2024-04-26T18:35:17.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 -- start start 2024-04-26T18:35:17.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b201034c0 0x7f2b2019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:17.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 0x7f2b2019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:17.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20113670 con 0x7f2b20109560 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201134f0 con 0x7f2b201034c0 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b2844b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201137f0 con 0x7f2b20103e80 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b269e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b269e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45008/0 (socket says 172.21.15.37:45008) 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b269e8700 1 -- 172.21.15.37:0/2933888940 learned_addr learned my addr 172.21.15.37:0/2933888940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b261e7700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b201034c0 0x7f2b2019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.599+0000 7f2b259e6700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 0x7f2b2019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 -- 172.21.15.37:0/2933888940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 msgr2=0x7f2b2019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 0x7f2b2019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 -- 172.21.15.37:0/2933888940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b201034c0 msgr2=0x7f2b2019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:17.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b201034c0 0x7f2b2019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 -- 172.21.15.37:0/2933888940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2019a030 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b269e8700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800e940 tx=0x7f2b1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800cbd0 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2019a320 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b1800cd70 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18005680 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.600+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201a6530 con 0x7f2b20109560 2024-04-26T18:35:17.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.601+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b20104bd0 con 0x7f2b20109560 2024-04-26T18:35:17.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.602+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b18004750 con 0x7f2b20109560 2024-04-26T18:35:17.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.603+0000 7f2b1f7fe700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 0x7f2b080a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:17.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.603+0000 7f2b261e7700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 0x7f2b080a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:17.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.603+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8837+0+0 (secure 0 0 0) 0x7f2b180d0de0 con 0x7f2b20109560 2024-04-26T18:35:17.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.603+0000 7f2b261e7700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 0x7f2b080a6270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e270 tx=0x7f2b10009040 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:17.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.606+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b18053ff0 con 0x7f2b20109560 2024-04-26T18:35:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:17.813+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2b2002e490 con 0x7f2b20109560 2024-04-26T18:35:18.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.218+0000 7f2b1f7fe700 1 -- 172.21.15.37:0/2933888940 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f2b1809d340 con 0x7f2b20109560 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 msgr2=0x7f2b080a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 0x7f2b080a6270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000e270 tx=0x7f2b10009040 comp rx=0 tx=0).stop 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 msgr2=0x7f2b20199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800e940 tx=0x7f2b1800ed00 comp rx=0 tx=0).stop 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 shutdown_connections 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b20109560 0x7f2b20199850 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:18.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f2b080a3db0 0x7f2b080a6270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:18.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b201034c0 0x7f2b2019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:18.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 --2- 172.21.15.37:0/2933888940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2b20103e80 0x7f2b2019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:18.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.222+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 >> 172.21.15.37:0/2933888940 conn(0x7f2b2007a5d0 msgr2=0x7f2b201020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:18.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.223+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 shutdown_connections 2024-04-26T18:35:18.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:18.223+0000 7f2b2844b700 1 -- 172.21.15.37:0/2933888940 wait complete. 2024-04-26T18:35:18.225 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-26T18:35:18.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:18 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:18.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:18 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:18.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:18 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:18.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid 755ef512-03f8-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T18:35:19.077 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:19.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:19 smithi037 ceph-mon[29171]: pgmap v344: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:19.456 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:19 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:19.456 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:19 smithi037 ceph-mon[29171]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:35:19.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:19 smithi120 ceph-mon[32244]: pgmap v344: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:19.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:19 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:19.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:19 smithi120 ceph-mon[32244]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:35:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:19 smithi170 ceph-mon[32697]: pgmap v344: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s 2024-04-26T18:35:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:19 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/2933888940' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:19.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:19 smithi170 ceph-mon[32697]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:35:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- 172.21.15.37:0/3426169094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 msgr2=0x7f86d406bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 --2- 172.21.15.37:0/3426169094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d406bfb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009990 tx=0x7f86d0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- 172.21.15.37:0/3426169094 shutdown_connections 2024-04-26T18:35:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 --2- 172.21.15.37:0/3426169094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d406c4f0 0x7f86d41102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 --2- 172.21.15.37:0/3426169094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d4109500 0x7f86d41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 --2- 172.21.15.37:0/3426169094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d406bfb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- 172.21.15.37:0/3426169094 >> 172.21.15.37:0/3426169094 conn(0x7f86d4079080 msgr2=0x7f86d4079490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- 172.21.15.37:0/3426169094 shutdown_connections 2024-04-26T18:35:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- 172.21.15.37:0/3426169094 wait complete. 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 Processor -- start 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.673+0000 7f86dbf47700 1 -- start start 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d406c4f0 0x7f86d4102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4109500 0x7f86d4103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41135e0 con 0x7f86d406bb30 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4113460 con 0x7f86d406c4f0 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86dbf47700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4113760 con 0x7f86d4109500 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45022/0 (socket says 172.21.15.37:45022) 2024-04-26T18:35:20.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d94e2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d406c4f0 0x7f86d4102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 -- 172.21.15.37:0/4031466394 learned_addr learned my addr 172.21.15.37:0/4031466394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86da4e4700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4109500 0x7f86d4103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 -- 172.21.15.37:0/4031466394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4109500 msgr2=0x7f86d4103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4109500 0x7f86d4103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 -- 172.21.15.37:0/4031466394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d406c4f0 msgr2=0x7f86d4102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d406c4f0 0x7f86d4102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 -- 172.21.15.37:0/4031466394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d41a8030 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.674+0000 7f86d9ce3700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d950 tx=0x7f86c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.675+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8016070 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.675+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c800fd40 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.675+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d41a8320 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.675+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800e3b0 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.675+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41a8830 con 0x7f86d406bb30 2024-04-26T18:35:20.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.676+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d406a110 con 0x7f86d406bb30 2024-04-26T18:35:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.677+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f86c8019070 con 0x7f86d406bb30 2024-04-26T18:35:20.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.678+0000 7f86c6ffd700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 0x7f86c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.678+0000 7f86d94e2700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 0x7f86c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.678+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8231+0+0 (secure 0 0 0) 0x7f86c80d0130 con 0x7f86d406bb30 2024-04-26T18:35:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.678+0000 7f86d94e2700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 0x7f86c00a6270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098e0 tx=0x7f86d0005960 comp rx=0 tx=0).ready entity=mgr.24820 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:20.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.681+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86c809c890 con 0x7f86d406bb30 2024-04-26T18:35:20.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:20.887+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86d402e490 con 0x7f86d406bb30 2024-04-26T18:35:21.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.222+0000 7f86c6ffd700 1 -- 172.21.15.37:0/4031466394 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7f86c809c190 con 0x7f86d406bb30 2024-04-26T18:35:21.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 msgr2=0x7f86c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:21.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 0x7f86c00a6270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098e0 tx=0x7f86d0005960 comp rx=0 tx=0).stop 2024-04-26T18:35:21.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 msgr2=0x7f86d4102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:21.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d950 tx=0x7f86c800dc60 comp rx=0 tx=0).stop 2024-04-26T18:35:21.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 shutdown_connections 2024-04-26T18:35:21.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86d4109500 0x7f86d4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:21.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86d406bb30 0x7f86d4102710 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:21.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.37:6800/1702580825,v1:172.21.15.37:6801/1702580825] conn(0x7f86c00a3db0 0x7f86c00a6270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:21.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 --2- 172.21.15.37:0/4031466394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f86d406c4f0 0x7f86d4102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:21.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.225+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 >> 172.21.15.37:0/4031466394 conn(0x7f86d4079080 msgr2=0x7f86d41012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:21.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.226+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 shutdown_connections 2024-04-26T18:35:21.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T18:35:21.226+0000 7f86dbf47700 1 -- 172.21.15.37:0/4031466394 wait complete. 2024-04-26T18:35:21.227 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-26T18:35:21.482 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:21 smithi037 ceph-mon[29171]: pgmap v346: 65 pgs: 65 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:35:21.483 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:21 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:21 smithi120 ceph-mon[32244]: pgmap v346: 65 pgs: 65 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:35:21.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:21 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:21.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:21 smithi170 ceph-mon[32697]: pgmap v346: 65 pgs: 65 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:35:21.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:21 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:35:21.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:35:21.962 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:35:21.991 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:35:22.018 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T18:35:22.018 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:35:22.035 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:35:22.049 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:35:22.073 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T18:35:22.073 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T18:35:22.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a 2024-04-26T18:35:22.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 systemd[1]: Stopping Ceph mon.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:22.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:22 smithi170 ceph-mon[32697]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:22.326 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:22 smithi170 ceph-mon[32697]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:35:22.502 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:22 smithi120 ceph-mon[32244]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:22.503 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:22 smithi120 ceph-mon[32244]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:35:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 ceph-mon[29171]: from='client.? 172.21.15.37:0/4031466394' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:35:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 ceph-mon[29171]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:35:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a[29167]: 2024-04-26T18:35:22.404+0000 7f5265a52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a[29167]: 2024-04-26T18:35:22.404+0000 7f5265a52700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T18:35:22.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 podman[114156]: 2024-04-26 18:35:22.455710881 +0000 UTC m=+0.167086862 container died 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:35:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 podman[114156]: 2024-04-26 18:35:22.854349958 +0000 UTC m=+0.565725940 container cleanup 631bb5bf4720e7e3c52b95f955a9249e12f700a8dbc1af11f79d36f32c02cce1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T18:35:23.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 18:35:22 smithi037 bash[114156]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-a 2024-04-26T18:35:23.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T18:35:23.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:23.266 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T18:35:23.266 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T18:35:23.266 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.b 2024-04-26T18:35:23.549 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:23 smithi120 systemd[1]: Stopping Ceph mon.b for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:24.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b[32240]: 2024-04-26T18:35:23.548+0000 7f21bcd3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:24.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b[32240]: 2024-04-26T18:35:23.548+0000 7f21bcd3f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T18:35:24.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:23 smithi120 podman[68826]: 2024-04-26 18:35:23.722363966 +0000 UTC m=+0.278929133 container stop 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:35:24.003 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:23 smithi120 podman[68826]: 2024-04-26 18:35:23.747881805 +0000 UTC m=+0.304446977 container died 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T18:35:24.454 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:24 smithi120 podman[68826]: 2024-04-26 18:35:24.098632868 +0000 UTC m=+0.655198034 container cleanup 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:35:24.455 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:24 smithi120 bash[68826]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b 2024-04-26T18:35:24.455 INFO:journalctl@ceph.mon.b.smithi120.stdout:Apr 26 18:35:24 smithi120 podman[68840]: 2024-04-26 18:35:24.258129589 +0000 UTC m=+0.526698635 container remove 2cbb34997323a2dda207a271a33fd0e361be866ce6d713fc3c5460a681b13cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T18:35:24.465 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T18:35:24.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:24.513 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T18:35:24.513 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T18:35:24.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.c 2024-04-26T18:35:24.816 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:24 smithi170 systemd[1]: Stopping Ceph mon.c for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:25.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c[32683]: 2024-04-26T18:35:24.816+0000 7fc245611700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:25.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c[32683]: 2024-04-26T18:35:24.816+0000 7fc245611700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T18:35:25.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:25 smithi170 podman[69531]: 2024-04-26 18:35:25.011018715 +0000 UTC m=+0.297401249 container died c1339d7ac19a371a6beb585ea6c3e5a22113d5cf4acd9cd57464e1897645c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:35:25.575 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:25 smithi170 podman[69531]: 2024-04-26 18:35:25.392227892 +0000 UTC m=+0.678610419 container cleanup c1339d7ac19a371a6beb585ea6c3e5a22113d5cf4acd9cd57464e1897645c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T18:35:25.576 INFO:journalctl@ceph.mon.c.smithi170.stdout:Apr 26 18:35:25 smithi170 bash[69531]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mon-c 2024-04-26T18:35:25.814 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T18:35:25.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:25.937 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T18:35:25.937 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T18:35:25.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x 2024-04-26T18:35:26.233 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:35:25 smithi037 systemd[1]: Stopping Ceph mgr.x for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:26.554 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:35:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x[29402]: 2024-04-26T18:35:26.234+0000 7fabd2c6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:26.554 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:35:26 smithi037 podman[114289]: 2024-04-26 18:35:26.266196239 +0000 UTC m=+0.135464387 container died 8b820eaa54519bc7e0407ecb8168a9f910f44f058bbb26df680ab839d72a2be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T18:35:26.877 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:35:26 smithi037 podman[114289]: 2024-04-26 18:35:26.626149593 +0000 UTC m=+0.495417732 container cleanup 8b820eaa54519bc7e0407ecb8168a9f910f44f058bbb26df680ab839d72a2be4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T18:35:26.877 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 18:35:26 smithi037 bash[114289]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-x 2024-04-26T18:35:26.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T18:35:27.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:27.023 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T18:35:27.023 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T18:35:27.023 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.y 2024-04-26T18:35:27.339 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:35:27 smithi120 systemd[1]: Stopping Ceph mgr.y for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:27.669 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:35:27 smithi120 podman[68948]: 2024-04-26 18:35:27.340484987 +0000 UTC m=+0.140034295 container died a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T18:35:27.980 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:35:27 smithi120 podman[68948]: 2024-04-26 18:35:27.77870269 +0000 UTC m=+0.578251986 container cleanup a21af0e1dac68d0106740aa7de5d87b422f34fdd523fcabf2af58074874659f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T18:35:27.980 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Apr 26 18:35:27 smithi120 bash[68948]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-y 2024-04-26T18:35:28.200 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T18:35:28.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:28.233 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T18:35:28.234 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T18:35:28.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.z 2024-04-26T18:35:28.575 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:35:28 smithi170 systemd[1]: Stopping Ceph mgr.z for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:28.863 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:35:28 smithi170 podman[69653]: 2024-04-26 18:35:28.576266348 +0000 UTC m=+0.145511125 container died 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:35:29.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:35:28 smithi170 podman[69653]: 2024-04-26 18:35:28.946913058 +0000 UTC m=+0.516157827 container cleanup 694f87fa27e94f0aa0a8a6c9252e8edcc177039f08994992895e11f8a083f366 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T18:35:29.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Apr 26 18:35:28 smithi170 bash[69653]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mgr-z 2024-04-26T18:35:29.358 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T18:35:29.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:29.391 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T18:35:29.391 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T18:35:29.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.0 2024-04-26T18:35:29.733 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:29 smithi037 systemd[1]: Stopping Ceph osd.0 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:30.232 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0[41296]: 2024-04-26T18:35:29.838+0000 7fb441d1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:30.233 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0[41296]: 2024-04-26T18:35:29.838+0000 7fb441d1e700 -1 osd.0 107 *** Got signal Terminated *** 2024-04-26T18:35:30.233 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0[41296]: 2024-04-26T18:35:29.838+0000 7fb441d1e700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:35:35.573 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:35 smithi037 podman[114410]: 2024-04-26 18:35:35.277383429 +0000 UTC m=+5.686056731 container died 07bf2db1e663377111bcf373ed16dc77fbbb40c5df4524b00a3b96a5afc4875b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T18:35:35.898 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:35 smithi037 podman[114410]: 2024-04-26 18:35:35.714276086 +0000 UTC m=+6.122949391 container cleanup 07bf2db1e663377111bcf373ed16dc77fbbb40c5df4524b00a3b96a5afc4875b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T18:35:35.898 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:35 smithi037 bash[114410]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0 2024-04-26T18:35:36.232 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:35 smithi037 podman[114429]: 2024-04-26 18:35:35.898844328 +0000 UTC m=+0.621308696 container remove 07bf2db1e663377111bcf373ed16dc77fbbb40c5df4524b00a3b96a5afc4875b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T18:35:36.619 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.22352246 +0000 UTC m=+0.018496974 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:35:36.619 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.324771898 +0000 UTC m=+0.119746454 container create 0bc4da88b0d9c3a2d04207a71deb7dd8130cc09f146382e3dd357befde5d74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T18:35:36.619 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.543732028 +0000 UTC m=+0.338706542 container init 0bc4da88b0d9c3a2d04207a71deb7dd8130cc09f146382e3dd357befde5d74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T18:35:36.619 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.546894618 +0000 UTC m=+0.341869140 container start 0bc4da88b0d9c3a2d04207a71deb7dd8130cc09f146382e3dd357befde5d74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:35:36.983 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.618681543 +0000 UTC m=+0.413656071 container attach 0bc4da88b0d9c3a2d04207a71deb7dd8130cc09f146382e3dd357befde5d74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T18:35:36.983 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 18:35:36 smithi037 podman[114524]: 2024-04-26 18:35:36.853939647 +0000 UTC m=+0.648914169 container died 0bc4da88b0d9c3a2d04207a71deb7dd8130cc09f146382e3dd357befde5d74f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T18:35:37.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T18:35:37.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:37.586 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T18:35:37.586 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T18:35:37.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.1 2024-04-26T18:35:37.882 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:37 smithi120 systemd[1]: Stopping Ceph osd.1 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:38.253 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1[36846]: 2024-04-26T18:35:37.881+0000 7f5fa9e2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:38.253 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1[36846]: 2024-04-26T18:35:37.881+0000 7f5fa9e2e700 -1 osd.1 107 *** Got signal Terminated *** 2024-04-26T18:35:38.253 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1[36846]: 2024-04-26T18:35:37.881+0000 7f5fa9e2e700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:35:43.558 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:43 smithi120 podman[69070]: 2024-04-26 18:35:43.26249273 +0000 UTC m=+5.489814014 container died 794c01f9683ba50d8bdd44c8f7e9baa4c4683e0e17f487643095280346775e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0) 2024-04-26T18:35:43.933 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:43 smithi120 podman[69070]: 2024-04-26 18:35:43.649124322 +0000 UTC m=+5.876445592 container cleanup 794c01f9683ba50d8bdd44c8f7e9baa4c4683e0e17f487643095280346775e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T18:35:43.934 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:43 smithi120 bash[69070]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1 2024-04-26T18:35:44.243 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:43 smithi120 podman[69086]: 2024-04-26 18:35:43.934420925 +0000 UTC m=+0.670377146 container remove 794c01f9683ba50d8bdd44c8f7e9baa4c4683e0e17f487643095280346775e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:35:44.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.146934537 +0000 UTC m=+0.017761431 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:35:44.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.252326062 +0000 UTC m=+0.123152956 container create 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:35:44.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.463522319 +0000 UTC m=+0.334349223 container init 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:35:44.503 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.466816653 +0000 UTC m=+0.337643550 container start 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2) 2024-04-26T18:35:44.758 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.52999365 +0000 UTC m=+0.400820557 container attach 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:35:45.057 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:44 smithi120 podman[69181]: 2024-04-26 18:35:44.758747621 +0000 UTC m=+0.629574508 container died 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T18:35:45.315 INFO:journalctl@ceph.osd.1.smithi120.stdout:Apr 26 18:35:45 smithi120 podman[69206]: 2024-04-26 18:35:45.300642864 +0000 UTC m=+0.530613351 container remove 87d4fe4448588b85bad48072ce85adf973a03f71b60b2f442130a82e59d7a996 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T18:35:45.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T18:35:45.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:45.361 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T18:35:45.361 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T18:35:45.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.2 2024-04-26T18:35:45.667 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:45 smithi170 systemd[1]: Stopping Ceph osd.2 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:45.923 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2[37832]: 2024-04-26T18:35:45.666+0000 7f82a6e32700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:45.924 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2[37832]: 2024-04-26T18:35:45.666+0000 7f82a6e32700 -1 osd.2 107 *** Got signal Terminated *** 2024-04-26T18:35:45.924 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2[37832]: 2024-04-26T18:35:45.666+0000 7f82a6e32700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:35:51.450 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:51 smithi170 podman[69778]: 2024-04-26 18:35:51.18095557 +0000 UTC m=+5.621905328 container died db70db4000a2ceae1fbd421c8d3e9d0db09c2c288800f80dea6c080a515239ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:35:51.800 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:51 smithi170 podman[69778]: 2024-04-26 18:35:51.541474056 +0000 UTC m=+5.982423845 container cleanup db70db4000a2ceae1fbd421c8d3e9d0db09c2c288800f80dea6c080a515239ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:35:51.800 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:51 smithi170 bash[69778]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2 2024-04-26T18:35:52.075 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:51 smithi170 podman[69807]: 2024-04-26 18:35:51.800804657 +0000 UTC m=+0.619939639 container remove db70db4000a2ceae1fbd421c8d3e9d0db09c2c288800f80dea6c080a515239ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:35:52.388 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.021253826 +0000 UTC m=+0.017747269 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:35:52.388 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.143397277 +0000 UTC m=+0.139890711 container create 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:35:52.689 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.388378298 +0000 UTC m=+0.384871736 container init 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T18:35:52.689 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.39196918 +0000 UTC m=+0.388462618 container start 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:35:52.689 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.50481071 +0000 UTC m=+0.501304168 container attach 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:35:53.057 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:52 smithi170 podman[69898]: 2024-04-26 18:35:52.68999868 +0000 UTC m=+0.686492109 container died 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:35:53.314 INFO:journalctl@ceph.osd.2.smithi170.stdout:Apr 26 18:35:53 smithi170 podman[69924]: 2024-04-26 18:35:53.300025808 +0000 UTC m=+0.603765336 container remove 4c2ad9e22f168ec2081da6fc7d683503b7432f67d4fcf6e364a2b340eecdd873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T18:35:53.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T18:35:53.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:35:53.359 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T18:35:53.359 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T18:35:53.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.3 2024-04-26T18:35:53.643 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:53 smithi037 systemd[1]: Stopping Ceph osd.3 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:35:53.911 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:53.643+0000 7feb99dcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:35:53.911 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:53.643+0000 7feb99dcb700 -1 osd.3 107 *** Got signal Terminated *** 2024-04-26T18:35:53.911 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:53 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:53.643+0000 7feb99dcb700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:35:56.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:35:55 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:35:55.964+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:35:56.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:35:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:35:55.828+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:35:56.732 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:56 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:56.481+0000 7feb963bf700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.245136+0000 front 2024-04-26T18:35:31.245195+0000 (oldest deadline 2024-04-26T18:35:55.945279+0000) 2024-04-26T18:35:56.752 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:35:56 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:35:56.358+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:35:56.781 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:35:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:35:56.396+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:35:57.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:35:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:35:56.781+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:35:57.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:35:56 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:35:56.941+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:35:57.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:35:56 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:35:56.926+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:35:57.752 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:35:57 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:35:57.364+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:35:57.764 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:35:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:35:57.374+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:35:57.952 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:57 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:57.509+0000 7feb963bf700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.245136+0000 front 2024-04-26T18:35:31.245195+0000 (oldest deadline 2024-04-26T18:35:55.945279+0000) 2024-04-26T18:35:58.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:35:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:35:57.764+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:35:58.232 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:35:57 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:35:57.952+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:35:58.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:35:57 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:35:57.978+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:35:58.746 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:35:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:35:58.414+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:35:58.752 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:35:58 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:35:58.389+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:35:58.908 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3[47844]: 2024-04-26T18:35:58.546+0000 7feb963bf700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.245136+0000 front 2024-04-26T18:35:31.245195+0000 (oldest deadline 2024-04-26T18:35:55.945279+0000) 2024-04-26T18:35:59.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:35:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:35:58.746+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:35:59.232 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:35:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:35:58.908+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:35:59.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:35:58 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:35:58.994+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:35:59.233 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:59 smithi037 podman[114642]: 2024-04-26 18:35:59.034522864 +0000 UTC m=+5.476284380 container died 10aad971f99ece13110b8de9d825112c6a3c068fb0e8756790523b9144c02bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:35:59.669 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:59 smithi037 podman[114642]: 2024-04-26 18:35:59.477051959 +0000 UTC m=+5.918813472 container cleanup 10aad971f99ece13110b8de9d825112c6a3c068fb0e8756790523b9144c02bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T18:35:59.669 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:59 smithi037 bash[114642]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3 2024-04-26T18:35:59.715 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:35:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:35:59.426+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:35:59.752 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:35:59 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:35:59.350+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:35:59.930 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:35:59 smithi037 podman[114672]: 2024-04-26 18:35:59.670149218 +0000 UTC m=+0.630860333 container remove 10aad971f99ece13110b8de9d825112c6a3c068fb0e8756790523b9144c02bc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:00.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:35:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:35:59.716+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:00.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:00.002+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:00.233 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.018773568 +0000 UTC m=+0.018844695 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:00.233 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.13771353 +0000 UTC m=+0.137784647 container create e46973c363ab010fd3b7fa02e769f359e25055aa09c6406dd7a98a129810e8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:36:00.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:35:59 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:35:59.928+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:00.646 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.340493777 +0000 UTC m=+0.340564895 container init e46973c363ab010fd3b7fa02e769f359e25055aa09c6406dd7a98a129810e8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T18:36:00.646 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.343648416 +0000 UTC m=+0.343719538 container start e46973c363ab010fd3b7fa02e769f359e25055aa09c6406dd7a98a129810e8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:00.647 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.398597132 +0000 UTC m=+0.398668262 container attach e46973c363ab010fd3b7fa02e769f359e25055aa09c6406dd7a98a129810e8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:36:00.647 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 18:36:00 smithi037 podman[114761]: 2024-04-26 18:36:00.646905329 +0000 UTC m=+0.646976455 container died e46973c363ab010fd3b7fa02e769f359e25055aa09c6406dd7a98a129810e8b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T18:36:00.753 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:00 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:00.354+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:00.755 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:00.421+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:00.982 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:00 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:00.879+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:01.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:00.756+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:01.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T18:36:01.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:01.261 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T18:36:01.261 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T18:36:01.261 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.4 2024-04-26T18:36:01.482 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:01 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:01.038+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:01.503 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:01 smithi120 systemd[1]: Stopping Ceph osd.4 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:01.503 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:01 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:01.337+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:01.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:01 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:01.168+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:01.768 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:01 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:01.411+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:02.003 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:01 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:01.577+0000 7f83c2527700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:02.003 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:01 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:01.577+0000 7f83c2527700 -1 osd.4 107 *** Got signal Terminated *** 2024-04-26T18:36:02.003 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:01 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:01.577+0000 7f83c2527700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:02.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:01 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:01.769+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:02.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:02.007+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:02.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:01 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:01.922+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:02.502 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:02 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:02.350+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:02.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:02 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:02.188+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:02.747 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:02 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:02.435+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:03.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:02 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:02.747+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:03.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:03 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:03.009+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:03.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:02 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:02.954+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:03.502 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:03 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:03.303+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:03.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:03 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:03.220+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:03.792 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:03 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:03.444+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:04.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:03 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:03.792+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:04.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:03 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:03.967+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:04.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:03 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:03.924+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:04.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:03 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:03.924+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:04.503 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:04 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:04.311+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:04.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:04 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:04.419+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:04.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:04 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:04.193+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:04.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:04 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:04.481+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:05.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:05.009+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:05.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:04.937+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:05.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:04.937+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:05.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:04 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:04.826+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:05.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:04 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:04.826+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:05.463 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:05 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:05.294+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:05.463 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:05 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:05.181+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:05.463 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:05 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:05.181+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:05.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:05 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:05.463+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:05.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:05 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:05.465+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:05.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:05 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:05.465+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:06.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:06.045+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:06.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:05.923+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:06.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:05 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:05.923+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:06.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:05 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:05.838+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:06.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:05 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:05.838+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:06.441 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:06 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4[43756]: 2024-04-26T18:36:06.311+0000 7f83bab13700 -1 osd.4 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:31.417636+0000 front 2024-04-26T18:35:31.417741+0000 (oldest deadline 2024-04-26T18:35:55.517581+0000) 2024-04-26T18:36:06.441 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:06 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:06.151+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:06.441 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:06 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:06.151+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:06.752 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:06 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:06.440+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:06.822 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:06.513+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:06.822 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:06.513+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:07.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:06.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:07.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:06 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:06.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:07.222 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:06 smithi120 podman[69297]: 2024-04-26 18:36:06.962052004 +0000 UTC m=+5.501194595 container died 4cc61f65380ababdf2297c11906f5c433e8f94bd885e2aa2bde8f33a5fdd2688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T18:36:07.222 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:07 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:07.104+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:07.222 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:07 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:07.104+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:07.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:07.045+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:07.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:07.045+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:07.232 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:06.966+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:07.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:06 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:06.966+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:07.466 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:07 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:07.120+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:07.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:07 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:07.401+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:07.503 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:07 smithi120 podman[69297]: 2024-04-26 18:36:07.40514782 +0000 UTC m=+5.944290419 container cleanup 4cc61f65380ababdf2297c11906f5c433e8f94bd885e2aa2bde8f33a5fdd2688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:07.503 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:07 smithi120 bash[69297]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4 2024-04-26T18:36:07.789 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:07 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:07.466+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:07.789 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:07 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:07.466+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:08.002 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:07 smithi120 podman[69316]: 2024-04-26 18:36:07.606450347 +0000 UTC m=+0.644215055 container remove 4cc61f65380ababdf2297c11906f5c433e8f94bd885e2aa2bde8f33a5fdd2688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:36:08.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:07 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:07.790+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:08.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:07 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:07.790+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:08.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:08.025+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:08.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:08.025+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:08.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:07.977+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:08.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:07 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:07.977+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:08.316 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:08.111+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:08.111+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:08.007051905 +0000 UTC m=+0.109981471 container create 255950a0d8cf41350fb38392dbcc2e25941be07dbcafd4a0384a578641430377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:07.91589897 +0000 UTC m=+0.018828518 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:08.234187254 +0000 UTC m=+0.337116786 container init 255950a0d8cf41350fb38392dbcc2e25941be07dbcafd4a0384a578641430377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:08.237266694 +0000 UTC m=+0.340196224 container start 255950a0d8cf41350fb38392dbcc2e25941be07dbcafd4a0384a578641430377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:36:08.317 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:08.317437076 +0000 UTC m=+0.420366602 container attach 255950a0d8cf41350fb38392dbcc2e25941be07dbcafd4a0384a578641430377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T18:36:08.485 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:08.099+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:08.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:08 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:08.356+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:08.753 INFO:journalctl@ceph.osd.4.smithi120.stdout:Apr 26 18:36:08 smithi120 podman[69404]: 2024-04-26 18:36:08.52641383 +0000 UTC m=+0.629343355 container died 255950a0d8cf41350fb38392dbcc2e25941be07dbcafd4a0384a578641430377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:36:08.792 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:08.486+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:08.793 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:08.486+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:09.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:08.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:09.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:08 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:08.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:09.103 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:09.077+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:09.103 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:09.077+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:09.129 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T18:36:09.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:09.166 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T18:36:09.166 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T18:36:09.166 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.5 2024-04-26T18:36:09.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:08.987+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:09.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:08.987+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:09.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:08.945+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:09.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:08 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:08.945+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:09.460 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:09.104+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:09.461 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 systemd[1]: Stopping Ceph osd.5 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:09.461 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:09.459+0000 7fa8e2457700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:09.461 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:09.459+0000 7fa8e2457700 -1 osd.5 107 *** Got signal Terminated *** 2024-04-26T18:36:09.752 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:09 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:09.406+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:09.802 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:09.459+0000 7fa8e2457700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:09.802 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:09.480+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:09.802 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:09.480+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:10.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:09.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:10.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:09 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:09.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:10.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:09.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:10.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:09.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:10.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:09.967+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:10.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:09.967+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:10.383 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:10.112+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:10.383 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:10.112+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:10.460 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:10.077+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:10.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:10 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:10.383+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:10.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:10.460+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:10.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:10.460+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:11.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:10 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:10.938+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:11.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:10 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:10.938+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:11.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:11 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:11.013+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:11.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:11 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:11.013+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:11.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:11.093+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:11.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:10.844+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:11.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:10 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:10.844+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:11.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:11.116+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:11.423 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:11.116+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:11.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:11 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:11.423+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:11.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:11.413+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:11.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:11.413+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:12.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:11 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:11.950+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:12.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:11 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:11.950+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:12.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:12.050+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:12.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:12.050+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:12.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:12.050+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:12.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:12.048+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:12.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:12.048+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:12.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:11.839+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:12.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:11.839+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:12.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:11 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:11.839+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:12.435 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:12.140+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:12.435 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:12.140+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:12.752 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:12 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:12.435+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:12.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:12.454+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:12.825 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:12.454+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:13.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:12.841+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:13.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:12.841+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:13.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:12 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:12.841+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:13.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:13.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:13.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:13.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:13.430 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:13.145+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:13.430 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:13.145+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:13.482 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:12.989+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:13.483 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:12.989+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:13.483 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:12.989+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:13.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:13.096+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:13.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:13.096+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:13.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:13.096+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:13.752 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:13.431+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:13.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:13 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:13.431+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:13.800 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:13.480+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:33.292278+0000 front 2024-04-26T18:35:33.292160+0000 (oldest deadline 2024-04-26T18:35:56.192409+0000) 2024-04-26T18:36:13.801 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5[44792]: 2024-04-26T18:36:13.480+0000 7fa8dea4b700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.395379+0000 front 2024-04-26T18:35:42.395307+0000 (oldest deadline 2024-04-26T18:36:04.695377+0000) 2024-04-26T18:36:14.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:13.800+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:14.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:13.800+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:14.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:13 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:13.800+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:14.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:13.957+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:14.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:13.957+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:14.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:13 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:13.957+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:14.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:14.056+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:14.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:14.056+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:14.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:14.056+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:14.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:14.385+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:14.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:14.385+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:14.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:14.177+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:14.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:14 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:14.177+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:14.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:14.121+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:14.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:14.121+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:15.077 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:14 smithi170 podman[70004]: 2024-04-26 18:36:14.994301189 +0000 UTC m=+5.635167365 container died a76da6ab02fa197ad7d9092a8186215017d50b17a4dbc23579571b3491926a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:15.078 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:14.762+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:15.078 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:14.762+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:15.078 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:14 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:14.762+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:15.343 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:15.155+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:15.344 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:15.155+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:15.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:15.142+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:15.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:15.142+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:15.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:15.142+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:15.482 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:14.986+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:15.483 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:14.986+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:15.483 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:14 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:14.986+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:15.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:15.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:15.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:15.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:15.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:15.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:15.627 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:15 smithi170 podman[70004]: 2024-04-26 18:36:15.377793497 +0000 UTC m=+6.018659680 container cleanup a76da6ab02fa197ad7d9092a8186215017d50b17a4dbc23579571b3491926a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:15.628 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:15 smithi170 bash[70004]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5 2024-04-26T18:36:15.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:15.420+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:15.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:15 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:15.420+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:15.963 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:15 smithi170 podman[70021]: 2024-04-26 18:36:15.628365852 +0000 UTC m=+0.627008721 container remove a76da6ab02fa197ad7d9092a8186215017d50b17a4dbc23579571b3491926a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:15.963 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:15.754+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:15.963 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:15.754+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:15.964 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:15 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:15.754+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:15.948+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:15.948+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:15 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:15.948+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:16.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:16.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:16.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:16.020+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:16.306 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:16.161+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:16.161+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:15 smithi170 podman[70122]: 2024-04-26 18:36:15.866692844 +0000 UTC m=+0.018637539 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:15 smithi170 podman[70122]: 2024-04-26 18:36:15.970840421 +0000 UTC m=+0.122785117 container create a76fa6ce153ba76f2ba250285d997022dc1be691953a3dbc02d3556c5500e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:16 smithi170 podman[70122]: 2024-04-26 18:36:16.206632985 +0000 UTC m=+0.358577690 container init a76fa6ce153ba76f2ba250285d997022dc1be691953a3dbc02d3556c5500e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:16 smithi170 podman[70122]: 2024-04-26 18:36:16.210011826 +0000 UTC m=+0.361956530 container start a76fa6ce153ba76f2ba250285d997022dc1be691953a3dbc02d3556c5500e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T18:36:16.307 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:16 smithi170 podman[70122]: 2024-04-26 18:36:16.306495052 +0000 UTC m=+0.458439750 container attach a76fa6ce153ba76f2ba250285d997022dc1be691953a3dbc02d3556c5500e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T18:36:16.393 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:16.113+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:16.393 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:16.113+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:16.393 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:16.113+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:16.575 INFO:journalctl@ceph.osd.5.smithi170.stdout:Apr 26 18:36:16 smithi170 podman[70122]: 2024-04-26 18:36:16.504933579 +0000 UTC m=+0.656878289 container died a76fa6ce153ba76f2ba250285d997022dc1be691953a3dbc02d3556c5500e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:16.678 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:16.393+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:16.679 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:16 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:16.393+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:17.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:16.727+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:17.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:16.727+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:17.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:16 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:16.727+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:17.102 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T18:36:17.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:17.139 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T18:36:17.139 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T18:36:17.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.6 2024-04-26T18:36:17.183 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:16.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:17.184 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:16.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:17.184 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:16 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:16.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:17.184 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.060+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:17.184 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.060+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:17.184 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.060+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:17.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:17.146+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:17.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:17.146+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:17.425 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:17.146+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:17.446 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 systemd[1]: Stopping Ceph osd.6 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:17.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:17.170+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:17.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:17.170+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:17.733 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.445+0000 7f0569e64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:17.733 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.445+0000 7f0569e64700 -1 osd.6 107 *** Got signal Terminated *** 2024-04-26T18:36:17.733 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:17.445+0000 7f0569e64700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:17.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:17.424+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:17.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:17 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:17.424+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:18.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:17.707+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:18.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:17.707+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:18.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:17 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:17.707+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:18.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:17.954+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:18.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:17.954+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:18.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:17.954+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:18.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:18.078+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:18.234 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:18.078+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:18.234 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:18.078+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:18.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:18.126+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:18.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:18.126+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:18.405 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:18.126+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:18.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:18.174+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:18.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:18.174+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:18.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:18.405+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:18.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:18 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:18.405+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:19.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:18.734+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:19.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:18.734+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:19.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:18 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:18.734+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:19.042+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:19.042+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:19.042+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:18.923+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:18.923+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:19.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:18 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:18.923+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:19.438 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:19 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:19.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:19.439 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:19 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:19.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:19.439 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:19 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:19.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:19.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:19.194+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:19.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:19.194+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:19.752 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:19 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:19.438+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:19.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:19 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:19.438+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:20.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:19.749+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:20.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:19.749+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:20.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:19.749+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:20.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:19 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:19.749+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:20.074+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:20.074+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:20.074+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:19.882+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:19.882+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:20.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:19.882+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:20.234 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:19 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:19.882+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:20.431 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:20.158+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:20.431 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:20.158+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:20.431 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:20.158+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:20.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:20.201+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:20.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:20.201+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:20.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:20.201+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:20.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:20.430+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:20.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:20.430+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:21.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:20.783+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:21.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:20.783+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:21.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:20.783+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:21.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:20 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:20.784+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:21.121 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:20.871+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:21.122 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:20.871+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:21.122 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:20.871+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:21.122 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:20 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:20.871+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:21.389 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:21 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:21.115+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:21.389 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:21 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:21.115+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:21.389 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:21 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:21.115+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:21.482 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:21.121+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:21.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:21.121+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:21.483 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:21.121+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:21.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:21.233+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:21.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:21.233+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:21.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:21.233+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:21.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:21 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:21.389+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:21.753 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:21 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:21.389+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:22.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:21.776+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:22.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:21.776+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:22.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:21.776+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:22.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:21 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:21.776+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:22.232 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:22.080+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.549026+0000 front 2024-04-26T18:35:34.549063+0000 (oldest deadline 2024-04-26T18:35:56.249247+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:22.080+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.950525+0000 front 2024-04-26T18:35:40.950742+0000 (oldest deadline 2024-04-26T18:36:03.850611+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6[54856]: 2024-04-26T18:36:22.080+0000 7f0562450700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.552809+0000 front 2024-04-26T18:35:49.552747+0000 (oldest deadline 2024-04-26T18:36:11.852758+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:21.865+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:21.865+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:21.865+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:22.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:21 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:21.865+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:22.502 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:22.139+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:22.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:22.139+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:22.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:22.139+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:22.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:22.342+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:22.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:22 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:22.342+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:22.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:22.228+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:22.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:22.228+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:22.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:22.228+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:23.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:22.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:23.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:22.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:23.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:22.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:23.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:22 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:22.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:23.134 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:22 smithi037 podman[114866]: 2024-04-26 18:36:22.927570524 +0000 UTC m=+5.591593758 container died fb64ddd809bd907837daeb6e47aa591d973802ae082660e875c42cd6630c3a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T18:36:23.135 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:22.872+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:23.135 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:22.872+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:23.135 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:22.872+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:23.135 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:22 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:22.872+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:23.466 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:23 smithi037 podman[114866]: 2024-04-26 18:36:23.262738182 +0000 UTC m=+5.926761411 container cleanup fb64ddd809bd907837daeb6e47aa591d973802ae082660e875c42cd6630c3a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:36:23.466 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:23 smithi037 bash[114866]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:23.315+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:23.315+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:23.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:23.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:23.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:23.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:23 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:23.124+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:23.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:23.204+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:23.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:23.204+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:23.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:23.204+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:23.732 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:23 smithi037 podman[114883]: 2024-04-26 18:36:23.463884322 +0000 UTC m=+0.539650561 container remove fb64ddd809bd907837daeb6e47aa591d973802ae082660e875c42cd6630c3a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T18:36:24.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:23.725+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:24.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:23.725+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:24.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:23.725+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:24.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:23 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:23.725+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:23 smithi037 podman[114974]: 2024-04-26 18:36:23.795484014 +0000 UTC m=+0.018474633 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:23 smithi037 podman[114974]: 2024-04-26 18:36:23.90650301 +0000 UTC m=+0.129493656 container create 672c23c676f0a4c7ab57f22e6766692beb1984b5630024528b5f65d9dfb00f9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:23.840+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:23.840+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:23.840+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:24.108 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:23 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:23.840+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:24.416 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:24 smithi037 podman[114974]: 2024-04-26 18:36:24.108688596 +0000 UTC m=+0.331679200 container init 672c23c676f0a4c7ab57f22e6766692beb1984b5630024528b5f65d9dfb00f9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T18:36:24.416 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:24 smithi037 podman[114974]: 2024-04-26 18:36:24.112085037 +0000 UTC m=+0.335075647 container start 672c23c676f0a4c7ab57f22e6766692beb1984b5630024528b5f65d9dfb00f9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:24.417 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:24 smithi037 podman[114974]: 2024-04-26 18:36:24.200321087 +0000 UTC m=+0.423311705 container attach 672c23c676f0a4c7ab57f22e6766692beb1984b5630024528b5f65d9dfb00f9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:24.273+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:24.273+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:24.273+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:24.094+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:24.094+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:24.094+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:24.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:24 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:24.094+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:24.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:24.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:24.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:24.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:24.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:24.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:24.702 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 18:36:24 smithi037 podman[114974]: 2024-04-26 18:36:24.417593262 +0000 UTC m=+0.640583871 container died 672c23c676f0a4c7ab57f22e6766692beb1984b5630024528b5f65d9dfb00f9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:36:24.953 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:24 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:24.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:24.953 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:24 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:24.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:24.953 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:24 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:24.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:24.953 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:24 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:24.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:24.977 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T18:36:25.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:25.058 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T18:36:25.058 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T18:36:25.059 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.7 2024-04-26T18:36:25.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:24.743+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:25.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:24.743+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:25.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:24.743+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:25.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:24 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:24.743+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:25.125+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:25.125+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:25.125+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:25.125+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 systemd[1]: Stopping Ceph osd.7 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.232+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.232+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:25.340 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.232+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:25.341 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.340+0000 7f34b60d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:25.341 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.340+0000 7f34b60d8700 -1 osd.7 107 *** Got signal Terminated *** 2024-04-26T18:36:25.341 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:25.340+0000 7f34b60d8700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:25.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:25.153+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:25.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:25.153+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:25.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:25.153+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:26.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:25.731+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:26.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:25.731+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:26.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:25.731+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:26.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:25 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:25.731+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:26.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:25.829+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:26.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:25.829+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:26.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:25.829+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:26.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:25 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:25.829+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:26.217+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:26.217+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:26.217+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:26.143+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:26.143+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:26.143+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:26.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:26 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:26.143+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:26.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:26.194+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:26.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:26.194+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:26.576 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:26.194+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:27.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:26.684+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:27.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:26.684+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:27.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:26.684+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:27.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:26 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:26.684+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:27.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:26.849+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:27.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:26.849+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:27.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:26.849+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:27.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:26 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:26.849+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:27.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:27.178+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:27.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:27.178+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:27.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:27.178+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:27.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:27.141+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:27.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:27.141+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:27.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:27.141+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:27.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:27 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:27.141+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:27.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:27.204+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:27.576 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:27.204+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:27.576 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:27.205+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:28.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:27.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:28.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:27.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:28.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:27.729+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:28.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:27 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:27.730+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:28.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:27.855+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:28.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:27.855+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:28.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:27.855+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:28.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:27 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:27.855+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:28.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:28.158+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:28.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:28.158+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:28.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:28.158+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:28.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:28.186+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:28.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:28.186+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:28.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:28.186+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:28.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:28 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:28.186+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:28.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:28.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:28.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:28.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:28.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:28.187+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:29.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:28.766+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:29.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:28.766+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:29.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:28.766+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:29.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:28 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:28.766+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:29.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:28.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:29.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:28.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:29.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:28.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:29.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:28.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:29.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:28 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:28.876+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:29.176+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:29.176+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:29.176+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:29.216+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:29.216+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:29.216+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:29.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:29.216+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:29.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:29.161+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:29.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:29.161+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:29.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:29.161+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:30.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:29.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:30.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:29.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:30.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:29.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:30.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:29.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:30.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:29 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:29.801+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:30.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:29.862+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:30.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:29.862+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:30.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:29.862+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:30.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:29.862+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:30.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:29 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:29.862+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:30.137+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.520137+0000 front 2024-04-26T18:35:40.520345+0000 (oldest deadline 2024-04-26T18:36:04.020110+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:30.137+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:48.121232+0000 front 2024-04-26T18:35:48.121413+0000 (oldest deadline 2024-04-26T18:36:12.721787+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-26T18:36:30.137+0000 7f34b26cc700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:58.623035+0000 front 2024-04-26T18:35:58.622858+0000 (oldest deadline 2024-04-26T18:36:23.322869+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:30.202+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:30.202+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:30.202+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:30.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:30.202+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:30.504 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:30 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:30.202+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:30.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:30.120+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:30.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:30.120+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:30.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:30.120+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:31.048 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:30 smithi120 podman[69522]: 2024-04-26 18:36:30.77337551 +0000 UTC m=+5.525777699 container died 6af3b49f75dc8016ec7c3909182241de6997eac3dbacd2666423bfa304e7c610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:31.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:30.785+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:31.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:30.785+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:31.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:30.785+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:31.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:30.785+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:31.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:30 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:30.785+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:31.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:30 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:30.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:31.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:30 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:30.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:31.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:30 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:30.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:31.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:30 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:30.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:31.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:30 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:30.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:31.400 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69522]: 2024-04-26 18:36:31.147782506 +0000 UTC m=+5.900184735 container cleanup 6af3b49f75dc8016ec7c3909182241de6997eac3dbacd2666423bfa304e7c610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:36:31.400 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 bash[69522]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7 2024-04-26T18:36:31.400 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:31 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:31.212+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:31.400 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:31 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:31.212+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:31.401 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:31 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:31.212+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:31.401 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:31 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:31.212+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:31.401 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:31 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:31.212+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:31.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:31.112+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:31.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:31.112+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:31.576 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:31.112+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:31.719 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69540]: 2024-04-26 18:36:31.400629037 +0000 UTC m=+0.630735418 container remove 6af3b49f75dc8016ec7c3909182241de6997eac3dbacd2666423bfa304e7c610 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T18:36:32.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69629]: 2024-04-26 18:36:31.62283753 +0000 UTC m=+0.017884058 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:32.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69629]: 2024-04-26 18:36:31.724498688 +0000 UTC m=+0.119545246 container create f8d425785540606714a53bb3590241769a95580cb9e439c2460ceecb2a9e5681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T18:36:32.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69629]: 2024-04-26 18:36:31.943531469 +0000 UTC m=+0.338577985 container init f8d425785540606714a53bb3590241769a95580cb9e439c2460ceecb2a9e5681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T18:36:32.003 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:31 smithi120 podman[69629]: 2024-04-26 18:36:31.946630865 +0000 UTC m=+0.341677386 container start f8d425785540606714a53bb3590241769a95580cb9e439c2460ceecb2a9e5681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T18:36:32.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:31.821+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:32.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:31.821+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:32.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:31.821+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:32.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:31.821+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:32.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:31 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:31.821+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:32.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:31.858+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:32.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:31.858+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:32.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:31.858+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:32.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:31.858+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:32.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:31 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:31.858+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:32 smithi120 podman[69629]: 2024-04-26 18:36:32.035639699 +0000 UTC m=+0.430686228 container attach f8d425785540606714a53bb3590241769a95580cb9e439c2460ceecb2a9e5681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.7.smithi120.stdout:Apr 26 18:36:32 smithi120 podman[69629]: 2024-04-26 18:36:32.238318422 +0000 UTC m=+0.633364930 container died f8d425785540606714a53bb3590241769a95580cb9e439c2460ceecb2a9e5681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:32 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:32.201+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:32 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:32.201+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:32 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:32.201+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:32 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:32.201+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:32.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:32 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:32.201+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:32.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:32.083+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:32.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:32.083+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:32.576 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:32.083+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:32.807 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T18:36:32.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:32.844 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T18:36:32.844 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T18:36:32.844 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.8 2024-04-26T18:36:33.145 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:32.864+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:33.145 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:32.864+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:33.145 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:32.864+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:33.145 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:32.864+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:32 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:32.864+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:32 smithi170 systemd[1]: Stopping Ceph osd.8 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.043+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.043+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.043+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.043+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.144+0000 7eff1cd4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.144+0000 7eff1cd4b700 -1 osd.8 107 *** Got signal Terminated *** 2024-04-26T18:36:33.146 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:33.144+0000 7eff1cd4b700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:33.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:32.812+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:33.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:32.812+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:33.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:32.812+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:33.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:32.812+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:33.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:32 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:32.812+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:33.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:33.217+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:33.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:33.217+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:33.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:33.217+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:33.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:33.217+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:33.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:33.217+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:34.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:33 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:33.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:34.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:33 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:33.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:34.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:33 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:33.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:34.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:33 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:33.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:34.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:33 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:33.838+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:34.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:33.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:33.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:33.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:33.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:33 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:33.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:34.074+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:34.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:34.074+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:34.327 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:34.074+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:34.327 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:34.074+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:34.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:34 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:34.234+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:34.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:34 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:34.234+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:34.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:34 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:34.234+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:34.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:34 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:34.234+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:34.503 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:34 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:34.234+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:35.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:34.852+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:35.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:34.852+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:35.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:34.852+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:35.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:34.852+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:35.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:34 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:34.852+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:35.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:35.060+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:35.060+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:35.060+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:35.060+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:34.849+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:34.849+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:34.849+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:34.849+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:35.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:34 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:34.849+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:35.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:35 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:35.276+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:35.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:35 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:35.276+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:35.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:35 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:35.276+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:35.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:35 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:35.276+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:35.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:35 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:35.276+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:36.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:35.836+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:36.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:35.836+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:36.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:35.836+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:36.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:35.836+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:36.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:35 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:35.836+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:36.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:35 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:35.885+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:36.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:35 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:35.885+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:36.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:35 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:35.885+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:36.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:35 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:35.885+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:36.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:35 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:35.885+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:36.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:36.089+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:36.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:36.089+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:36.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:36.089+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:36.575 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:36.089+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:36.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:36 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:36.316+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:37.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:36 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:36.879+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:37.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:36 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:36.879+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:37.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:36 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:36.879+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:37.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:36 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:36.879+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:37.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:36 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:36.879+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:37.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:37.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:37.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:37.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:37.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:37.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:37.090+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:12.476087+0000 front 2024-04-26T18:36:12.476134+0000 (oldest deadline 2024-04-26T18:36:36.576281+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:36.867+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:36.867+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:36.867+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:36.867+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:37.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:36 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:36.867+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:37.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:37 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:37.303+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:38.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:37 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:37.900+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:38.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:37 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:37.900+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:38.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:37 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:37.900+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:38.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:37 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:37.900+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:38.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:37 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:37.900+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:38.069+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:42.072398+0000 front 2024-04-26T18:35:42.072455+0000 (oldest deadline 2024-04-26T18:36:06.772324+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:38.069+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.772589+0000 front 2024-04-26T18:35:46.772645+0000 (oldest deadline 2024-04-26T18:36:11.973211+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:38.069+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.074548+0000 front 2024-04-26T18:35:56.074461+0000 (oldest deadline 2024-04-26T18:36:19.574303+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:38.069+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.575941+0000 front 2024-04-26T18:36:06.575717+0000 (oldest deadline 2024-04-26T18:36:32.475764+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8[51513]: 2024-04-26T18:36:38.069+0000 7eff1933f700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:12.476087+0000 front 2024-04-26T18:36:12.476134+0000 (oldest deadline 2024-04-26T18:36:36.576281+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:37.854+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:37.854+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:38.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:37.854+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:38.327 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:37.854+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:38.327 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:37 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:37.854+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:38.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:38 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:38.325+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:38.919 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 podman[70229]: 2024-04-26 18:36:38.629843643 +0000 UTC m=+5.583525970 container died 09a3f0d85d2d6d253dcd8b48d2510580d27bf1c7c9a3c4c6105a743d706ed4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T18:36:38.919 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:38.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:38.920 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:38.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:38.920 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:38.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:38.920 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:38.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:38.920 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:38.869+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:39.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:38 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:38.902+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:39.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:38 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:38.902+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:39.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:38 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:38.902+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:39.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:38 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:38.902+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:39.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:38 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:38.902+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:39.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 podman[70229]: 2024-04-26 18:36:38.920026787 +0000 UTC m=+5.873709115 container cleanup 09a3f0d85d2d6d253dcd8b48d2510580d27bf1c7c9a3c4c6105a743d706ed4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:36:39.325 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:38 smithi170 bash[70229]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8 2024-04-26T18:36:39.326 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70244]: 2024-04-26 18:36:39.146227583 +0000 UTC m=+0.516197155 container remove 09a3f0d85d2d6d253dcd8b48d2510580d27bf1c7c9a3c4c6105a743d706ed4b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:39.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:39 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:39.298+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:39.773 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.364713834 +0000 UTC m=+0.017994759 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:39.773 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.463663322 +0000 UTC m=+0.116944285 container create 202fdcc946a5163dfcf427978ecce6262d8903bb7b8c755cdbf84f058a4aa6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:36:39.773 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.69172061 +0000 UTC m=+0.345001528 container init 202fdcc946a5163dfcf427978ecce6262d8903bb7b8c755cdbf84f058a4aa6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-26T18:36:39.774 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.695128719 +0000 UTC m=+0.348409645 container start 202fdcc946a5163dfcf427978ecce6262d8903bb7b8c755cdbf84f058a4aa6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:39.774 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.774060433 +0000 UTC m=+0.427341362 container attach 202fdcc946a5163dfcf427978ecce6262d8903bb7b8c755cdbf84f058a4aa6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:40.075 INFO:journalctl@ceph.osd.8.smithi170.stdout:Apr 26 18:36:39 smithi170 podman[70339]: 2024-04-26 18:36:39.984128211 +0000 UTC m=+0.637409130 container died 202fdcc946a5163dfcf427978ecce6262d8903bb7b8c755cdbf84f058a4aa6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:40.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:39 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:39.823+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:40.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:39 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:39.927+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:40.605 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T18:36:40.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:40.645 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T18:36:40.645 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T18:36:40.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.9 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:40.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:40 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:40.320+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:40.859 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:40 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:40.777+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:40.956 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 systemd[1]: Stopping Ceph osd.9 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:40.956 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:40.956 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:40.956 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:40.957 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:40.957 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:40.957 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.899+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:41.232 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.955+0000 7fed46914700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:41.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.955+0000 7fed46914700 -1 osd.9 107 *** Got signal Terminated *** 2024-04-26T18:36:41.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:40 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:40.955+0000 7fed46914700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:41.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:41 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:41.302+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:42.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:42.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:42.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:42.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:42.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:42.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:41.751+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:42.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:41 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:41.875+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:42.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:42 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:42.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:43.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:43.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:43.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:43.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:43.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:43.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:42 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:42.786+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:43.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:42 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:42.911+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:43.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:43 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:43.315+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:44.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:44.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:44.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:44.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:44.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:44.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:43 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:43.789+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:44.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:43 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:43.945+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:44.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:44.754 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:44 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:44.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:45.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:45.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:45.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:45.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:45.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:45.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:44 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:44.793+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:45.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:44 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:44.936+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:45.654 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:45.654 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:45.654 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:45.655 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:45.655 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:45.655 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:45.655 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:45 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:45.333+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:46.075 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:46.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:46.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:46.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:46.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:46.076 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:45.815+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:34.194458+0000 front 2024-04-26T18:35:34.194485+0000 (oldest deadline 2024-04-26T18:35:55.294612+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.696647+0000 front 2024-04-26T18:35:41.696579+0000 (oldest deadline 2024-04-26T18:36:06.396735+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:46.397246+0000 front 2024-04-26T18:35:46.397226+0000 (oldest deadline 2024-04-26T18:36:12.297347+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:56.899069+0000 front 2024-04-26T18:35:56.899033+0000 (oldest deadline 2024-04-26T18:36:19.799118+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.501225+0000 front 2024-04-26T18:36:05.501153+0000 (oldest deadline 2024-04-26T18:36:28.401092+0000) 2024-04-26T18:36:46.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:45 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9[61471]: 2024-04-26T18:36:45.889+0000 7fed42f08700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:13.702301+0000 front 2024-04-26T18:36:13.702082+0000 (oldest deadline 2024-04-26T18:36:39.601971+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:46.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:46 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:46.324+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:46.823 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:46 smithi037 podman[115092]: 2024-04-26 18:36:46.519405487 +0000 UTC m=+5.671192259 container died 3294eec250f95f6eba7a9446caabee993e46e2d509e64e3ee9aaffc43c0fdfd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:36:47.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:46 smithi037 podman[115092]: 2024-04-26 18:36:46.914473194 +0000 UTC m=+6.066259977 container cleanup 3294eec250f95f6eba7a9446caabee993e46e2d509e64e3ee9aaffc43c0fdfd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T18:36:47.233 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:46 smithi037 bash[115092]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9 2024-04-26T18:36:47.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:47.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:47.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:47.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:47.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:47.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:46 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:46.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:47.592 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115109]: 2024-04-26 18:36:47.248692121 +0000 UTC m=+0.726117943 container remove 3294eec250f95f6eba7a9446caabee993e46e2d509e64e3ee9aaffc43c0fdfd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:47.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:47 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:47.323+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:47.843 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115196]: 2024-04-26 18:36:47.496184139 +0000 UTC m=+0.018457169 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:47.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115196]: 2024-04-26 18:36:47.599569351 +0000 UTC m=+0.121842395 container create 3301ba8cce5c526bfa2fd16072ad3dc2488ca395cb4dfe0d82dd70675fabe4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:36:47.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115196]: 2024-04-26 18:36:47.785557532 +0000 UTC m=+0.307830539 container init 3301ba8cce5c526bfa2fd16072ad3dc2488ca395cb4dfe0d82dd70675fabe4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T18:36:47.844 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115196]: 2024-04-26 18:36:47.788677573 +0000 UTC m=+0.310950572 container start 3301ba8cce5c526bfa2fd16072ad3dc2488ca395cb4dfe0d82dd70675fabe4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:48.099 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:47 smithi037 podman[115196]: 2024-04-26 18:36:47.843582557 +0000 UTC m=+0.365855556 container attach 3301ba8cce5c526bfa2fd16072ad3dc2488ca395cb4dfe0d82dd70675fabe4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:48.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:47 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:47.870+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:48.396 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 18:36:48 smithi037 podman[115196]: 2024-04-26 18:36:48.100090863 +0000 UTC m=+0.622363875 container died 3301ba8cce5c526bfa2fd16072ad3dc2488ca395cb4dfe0d82dd70675fabe4c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:36:48.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T18:36:48.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:48.730 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T18:36:48.730 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T18:36:48.731 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.10 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:48.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:48.306+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:49.039 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:48 smithi120 systemd[1]: Stopping Ceph osd.10 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:49.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:49.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:49.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:49.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:49.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:49.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:48 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:48.907+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:49.346 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.039+0000 7fe593ce8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:49.347 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.039+0000 7fe593ce8700 -1 osd.10 107 *** Got signal Terminated *** 2024-04-26T18:36:49.347 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.039+0000 7fe593ce8700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:49.347 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:49.347 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:49.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:49.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:49.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:49.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:49.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:49 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:49.346+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:50.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:50.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:50.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:50.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:50.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:50.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:49.948+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:50.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:50 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:50.321+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:51.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:51.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:51.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:51.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:51.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:51.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:50 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:50.933+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:51.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:51 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:51.272+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:52.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:52.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:52.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:52.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:52.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:52.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:51 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:51.903+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:52.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:52 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:52.307+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:53.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:53.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:53.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:53.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:53.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:53.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:53.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:52 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:52.919+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.124586+0000 front 2024-04-26T18:35:32.124510+0000 (oldest deadline 2024-04-26T18:36:00.325319+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:40.325826+0000 front 2024-04-26T18:35:40.325806+0000 (oldest deadline 2024-04-26T18:36:05.025976+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:49.027879+0000 front 2024-04-26T18:35:49.027993+0000 (oldest deadline 2024-04-26T18:36:14.927639+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:57.828627+0000 front 2024-04-26T18:35:57.828761+0000 (oldest deadline 2024-04-26T18:36:22.528620+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:06.430490+0000 front 2024-04-26T18:36:06.430515+0000 (oldest deadline 2024-04-26T18:36:29.831305+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:09.832056+0000 front 2024-04-26T18:36:09.832074+0000 (oldest deadline 2024-04-26T18:36:36.232358+0000) 2024-04-26T18:36:53.753 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:53 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10[57509]: 2024-04-26T18:36:53.304+0000 7fe5902dc700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T18:36:17.333172+0000 front 2024-04-26T18:36:17.333242+0000 (oldest deadline 2024-04-26T18:36:43.233071+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:54.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:53 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:53.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:54.731 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:54 smithi120 podman[69733]: 2024-04-26 18:36:54.464502682 +0000 UTC m=+5.536673288 container died e67063732d9fb39382ee7972041f142b95a3d1bb6a356eea94737c55792fd88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T18:36:55.002 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:54 smithi120 podman[69733]: 2024-04-26 18:36:54.814209136 +0000 UTC m=+5.886379745 container cleanup e67063732d9fb39382ee7972041f142b95a3d1bb6a356eea94737c55792fd88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:36:55.003 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:54 smithi120 bash[69733]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10 2024-04-26T18:36:55.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:55.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:54 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:54.922+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:55.415 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69753]: 2024-04-26 18:36:55.098945128 +0000 UTC m=+0.631912300 container remove e67063732d9fb39382ee7972041f142b95a3d1bb6a356eea94737c55792fd88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T18:36:55.726 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.416223146 +0000 UTC m=+0.094273464 container create b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:36:55.726 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.339455197 +0000 UTC m=+0.017505547 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:36:55.726 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.644027265 +0000 UTC m=+0.322077592 container init b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T18:36:55.726 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.647613259 +0000 UTC m=+0.325663596 container start b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T18:36:55.726 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.726692004 +0000 UTC m=+0.404742331 container attach b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:56.002 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:55 smithi120 podman[69843]: 2024-04-26 18:36:55.938872094 +0000 UTC m=+0.616922422 container died b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T18:36:56.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:56.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:55 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:55.915+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:56.497 INFO:journalctl@ceph.osd.10.smithi120.stdout:Apr 26 18:36:56 smithi120 podman[69869]: 2024-04-26 18:36:56.479775646 +0000 UTC m=+0.531344330 container remove b21fbc6184ff0a4e5baeebb02598b446a0e3d88b29e0567d0fea0d5bc1aa24d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:36:56.503 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T18:36:56.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:36:56.580 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T18:36:56.580 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T18:36:56.580 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.11 2024-04-26T18:36:56.879 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 systemd[1]: Stopping Ceph osd.11 for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:36:57.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.879+0000 7fa55f3e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.879+0000 7fa55f3e0700 -1 osd.11 107 *** Got signal Terminated *** 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.879+0000 7fa55f3e0700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:57.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:56 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:56.905+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:58.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:58.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:57 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:57.924+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:36:59.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:36:59.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:36:59.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:36:59.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:36:59.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:36:59.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:36:59.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:58 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:58.898+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:37:00.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:37:00.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:37:00.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:37:00.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:37:00.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:37:00.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:37:00.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:36:59 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:36:59.865+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:37:01.325 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T18:35:32.177465+0000 front 2024-04-26T18:35:32.177428+0000 (oldest deadline 2024-04-26T18:35:55.677501+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-04-26T18:35:41.380145+0000 front 2024-04-26T18:35:41.380159+0000 (oldest deadline 2024-04-26T18:36:04.279866+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-04-26T18:35:45.380825+0000 front 2024-04-26T18:35:45.380972+0000 (oldest deadline 2024-04-26T18:36:11.280911+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T18:35:55.882487+0000 front 2024-04-26T18:35:55.882394+0000 (oldest deadline 2024-04-26T18:36:19.382329+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-04-26T18:36:05.283382+0000 front 2024-04-26T18:36:05.283421+0000 (oldest deadline 2024-04-26T18:36:28.783364+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-04-26T18:36:14.084302+0000 front 2024-04-26T18:36:14.084336+0000 (oldest deadline 2024-04-26T18:36:39.284748+0000) 2024-04-26T18:37:01.326 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:00 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11[58093]: 2024-04-26T18:37:00.896+0000 7fa5579cc700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-04-26T18:36:29.687498+0000 front 2024-04-26T18:36:29.687440+0000 (oldest deadline 2024-04-26T18:36:52.587083+0000) 2024-04-26T18:37:02.479 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:02 smithi170 podman[70458]: 2024-04-26 18:37:02.202274303 +0000 UTC m=+5.426952150 container died e2fdb0f83876a1f94e1d145634f5256b55531cd9dd8e965123cbf9910f0f4911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:37:02.825 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:02 smithi170 podman[70458]: 2024-04-26 18:37:02.554231983 +0000 UTC m=+5.778909862 container cleanup e2fdb0f83876a1f94e1d145634f5256b55531cd9dd8e965123cbf9910f0f4911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:37:02.826 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:02 smithi170 bash[70458]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11 2024-04-26T18:37:03.163 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:02 smithi170 podman[70478]: 2024-04-26 18:37:02.846986349 +0000 UTC m=+0.638142190 container remove e2fdb0f83876a1f94e1d145634f5256b55531cd9dd8e965123cbf9910f0f4911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:37:03.465 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.164256693 +0000 UTC m=+0.114507953 container create 9264f61b069b04230339856a0ef9b979f9e7ae643cb501103cc76d1b663257a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T18:37:03.466 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.067962717 +0000 UTC m=+0.018213989 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:37:03.466 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.383177037 +0000 UTC m=+0.333428293 container init 9264f61b069b04230339856a0ef9b979f9e7ae643cb501103cc76d1b663257a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:37:03.466 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.386122642 +0000 UTC m=+0.336373898 container start 9264f61b069b04230339856a0ef9b979f9e7ae643cb501103cc76d1b663257a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:37:03.466 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.466380378 +0000 UTC m=+0.416631643 container attach 9264f61b069b04230339856a0ef9b979f9e7ae643cb501103cc76d1b663257a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T18:37:03.825 INFO:journalctl@ceph.osd.11.smithi170.stdout:Apr 26 18:37:03 smithi170 podman[70568]: 2024-04-26 18:37:03.680010074 +0000 UTC m=+0.630261331 container died 9264f61b069b04230339856a0ef9b979f9e7ae643cb501103cc76d1b663257a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:37:04.255 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T18:37:04.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:04.291 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T18:37:04.292 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T18:37:04.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.a 2024-04-26T18:37:04.606 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:04 smithi037 systemd[1]: Stopping Ceph mds.a for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:04.878 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a[68463]: 2024-04-26T18:37:04.606+0000 7f09593f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:04.878 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:04 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a[68463]: 2024-04-26T18:37:04.606+0000 7f09593f5700 -1 mds.a *** got signal Terminated *** 2024-04-26T18:37:08.881 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:08 smithi037 podman[115301]: 2024-04-26 18:37:08.618337567 +0000 UTC m=+4.124214112 container died f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:37:09.148 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:08 smithi037 podman[115301]: 2024-04-26 18:37:08.964342264 +0000 UTC m=+4.470218815 container cleanup f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:37:09.148 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:08 smithi037 bash[115301]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a 2024-04-26T18:37:09.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T18:37:09.420 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:09 smithi037 podman[115327]: 2024-04-26 18:37:09.149095064 +0000 UTC m=+0.525821046 container remove f30d4c996070915ba0288497fdcce129789b1c61ead5e8f4a77ea142eaa310f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T18:37:09.420 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:09 smithi037 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.a.service: Deactivated successfully. 2024-04-26T18:37:09.420 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:09 smithi037 systemd[1]: Stopped Ceph mds.a for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:37:09.421 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 18:37:09 smithi037 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.a.service: Consumed 1.125s CPU time. 2024-04-26T18:37:09.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:09.434 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T18:37:09.434 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T18:37:09.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.d 2024-04-26T18:37:09.733 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:09 smithi037 systemd[1]: Stopping Ceph mds.d for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:10.233 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d[67250]: 2024-04-26T18:37:09.765+0000 7fc986c9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:10.233 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:09 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d[67250]: 2024-04-26T18:37:09.765+0000 7fc986c9d700 -1 mds.d *** got signal Terminated *** 2024-04-26T18:37:11.702 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:11 smithi037 podman[115434]: 2024-04-26 18:37:11.445601299 +0000 UTC m=+1.787281541 container died a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T18:37:11.983 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:11 smithi037 podman[115434]: 2024-04-26 18:37:11.801814741 +0000 UTC m=+2.143494973 container cleanup a4f4b1ef543c8130f7657c53e12a19cfce84c58b13ce606fe244bec4fd18f052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T18:37:11.983 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 18:37:11 smithi037 bash[115434]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-d 2024-04-26T18:37:12.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T18:37:12.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:12.261 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T18:37:12.262 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T18:37:12.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.g 2024-04-26T18:37:12.580 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:12 smithi037 systemd[1]: Stopping Ceph mds.g for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:12.982 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g[68040]: 2024-04-26T18:37:12.579+0000 7f2f059d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:12.983 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:12 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g[68040]: 2024-04-26T18:37:12.579+0000 7f2f059d7700 -1 mds.g *** got signal Terminated *** 2024-04-26T18:37:16.904 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:16 smithi037 podman[115554]: 2024-04-26 18:37:16.593313682 +0000 UTC m=+4.115424231 container died 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:37:16.905 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:16 smithi037 podman[115554]: 2024-04-26 18:37:16.905232879 +0000 UTC m=+4.427343427 container cleanup 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T18:37:17.232 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:16 smithi037 bash[115554]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g 2024-04-26T18:37:17.233 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 18:37:17 smithi037 podman[115568]: 2024-04-26 18:37:17.098271486 +0000 UTC m=+0.505003696 container remove 0dc2a5a0550db2c894450697f063bec6cd1962f4b5191528baa1f5529393b7f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:37:17.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T18:37:17.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:17.379 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T18:37:17.379 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T18:37:17.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.j 2024-04-26T18:37:17.694 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:17 smithi037 systemd[1]: Stopping Ceph mds.j for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:17.982 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j[66786]: 2024-04-26T18:37:17.692+0000 7f53e1ed7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:17.983 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:17 smithi037 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j[66786]: 2024-04-26T18:37:17.692+0000 7f53e1ed7700 -1 mds.j *** got signal Terminated *** 2024-04-26T18:37:19.209 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:18 smithi037 podman[115676]: 2024-04-26 18:37:18.942113908 +0000 UTC m=+1.346438278 container died 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:37:19.482 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:19 smithi037 podman[115676]: 2024-04-26 18:37:19.30123447 +0000 UTC m=+1.705558893 container cleanup 6a25cf689be6193ad202acc23e33d8dbf3610627b0cf464f7b0bf43dfefba8d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:37:19.483 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 18:37:19 smithi037 bash[115676]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-j 2024-04-26T18:37:19.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T18:37:19.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:19.768 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T18:37:19.768 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T18:37:19.768 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.b 2024-04-26T18:37:20.155 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:19 smithi120 systemd[1]: Stopping Ceph mds.b for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:20.414 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: 2024-04-26T18:37:20.154+0000 7f6c3085a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:20.414 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:20 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b[62909]: 2024-04-26T18:37:20.154+0000 7f6c3085a700 -1 mds.b *** got signal Terminated *** 2024-04-26T18:37:24.503 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:24 smithi120 podman[69948]: 2024-04-26 18:37:24.163024351 +0000 UTC m=+4.195930178 container died c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T18:37:24.996 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:24 smithi120 podman[69948]: 2024-04-26 18:37:24.754804419 +0000 UTC m=+4.787710247 container cleanup c1cdb629633fa33541dbdd7508877d61fe40a7e96b44877c3a72bacb35b9e11c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:37:24.996 INFO:journalctl@ceph.mds.b.smithi120.stdout:Apr 26 18:37:24 smithi120 bash[69948]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-b 2024-04-26T18:37:25.128 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T18:37:25.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:25.163 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T18:37:25.163 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T18:37:25.163 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.e 2024-04-26T18:37:25.252 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:25 smithi120 systemd[1]: Stopping Ceph mds.e for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:25.753 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e[62277]: 2024-04-26T18:37:25.482+0000 7f7dfb4e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:25.753 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:25 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e[62277]: 2024-04-26T18:37:25.482+0000 7f7dfb4e0700 -1 mds.e *** got signal Terminated *** 2024-04-26T18:37:28.503 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:28 smithi120 podman[70080]: 2024-04-26 18:37:28.244045631 +0000 UTC m=+2.864790417 container died cf4c59280a5f648f32ab44a79386addb0dd7bb565b328575631773fefc48f2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:37:28.866 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:28 smithi120 podman[70080]: 2024-04-26 18:37:28.649507126 +0000 UTC m=+3.270251893 container cleanup cf4c59280a5f648f32ab44a79386addb0dd7bb565b328575631773fefc48f2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:37:28.866 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:28 smithi120 bash[70080]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e 2024-04-26T18:37:29.095 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T18:37:29.116 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:28 smithi120 podman[70092]: 2024-04-26 18:37:28.867084315 +0000 UTC m=+0.624576653 container remove cf4c59280a5f648f32ab44a79386addb0dd7bb565b328575631773fefc48f2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:37:29.116 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:29 smithi120 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-26T18:37:29.116 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:29 smithi120 systemd[1]: Stopped Ceph mds.e for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:37:29.116 INFO:journalctl@ceph.mds.e.smithi120.stdout:Apr 26 18:37:29 smithi120 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.e.service: Consumed 2.130s CPU time. 2024-04-26T18:37:29.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:29.128 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T18:37:29.128 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T18:37:29.128 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.h 2024-04-26T18:37:29.452 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:29 smithi120 systemd[1]: Stopping Ceph mds.h for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:29.752 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: 2024-04-26T18:37:29.452+0000 7f4d7148f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:29.753 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:29 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h[62598]: 2024-04-26T18:37:29.452+0000 7f4d7148f700 -1 mds.h *** got signal Terminated *** 2024-04-26T18:37:32.552 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:32 smithi120 podman[70201]: 2024-04-26 18:37:32.253258642 +0000 UTC m=+2.905944241 container died f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:37:32.552 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:32 smithi120 podman[70201]: 2024-04-26 18:37:32.553197501 +0000 UTC m=+3.205883100 container cleanup f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:37:32.886 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:32 smithi120 bash[70201]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h 2024-04-26T18:37:32.887 INFO:journalctl@ceph.mds.h.smithi120.stdout:Apr 26 18:37:32 smithi120 podman[70215]: 2024-04-26 18:37:32.787353065 +0000 UTC m=+0.534323508 container remove f68f429048b7044d2df0e97ade9a56f876cdc294d87def0784a6837eb434468b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:37:32.903 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T18:37:32.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:32.933 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T18:37:32.934 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T18:37:32.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.k 2024-04-26T18:37:33.247 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:33 smithi120 systemd[1]: Stopping Ceph mds.k for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:33.502 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-k[61958]: 2024-04-26T18:37:33.247+0000 7f5810922700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:33.503 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:33 smithi120 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-k[61958]: 2024-04-26T18:37:33.247+0000 7f5810922700 -1 mds.k *** got signal Terminated *** 2024-04-26T18:37:37.514 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:37 smithi120 podman[70323]: 2024-04-26 18:37:37.254977388 +0000 UTC m=+4.105600780 container died f7106543ce3d7d32111a1d033e100853adb144ae1e14b491f64528b01900221e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:37:37.872 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:37 smithi120 podman[70323]: 2024-04-26 18:37:37.606054184 +0000 UTC m=+4.456677611 container cleanup f7106543ce3d7d32111a1d033e100853adb144ae1e14b491f64528b01900221e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:37:37.873 INFO:journalctl@ceph.mds.k.smithi120.stdout:Apr 26 18:37:37 smithi120 bash[70323]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-k 2024-04-26T18:37:38.038 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T18:37:38.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:38.069 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T18:37:38.069 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T18:37:38.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.c 2024-04-26T18:37:38.543 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:38 smithi170 systemd[1]: Stopping Ceph mds.c for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:38.799 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: 2024-04-26T18:37:38.543+0000 7faa4340e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:38.799 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:38 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c[62997]: 2024-04-26T18:37:38.543+0000 7faa4340e700 -1 mds.c *** got signal Terminated *** 2024-04-26T18:37:40.485 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:40 smithi170 podman[70674]: 2024-04-26 18:37:40.23521579 +0000 UTC m=+1.977131053 container died bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:37:40.759 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:40 smithi170 podman[70674]: 2024-04-26 18:37:40.576252463 +0000 UTC m=+2.318167723 container cleanup bc916e20a0de91d5d5e852d64c495498f2b19f69928ec124bf73b74c1b82653f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:37:40.760 INFO:journalctl@ceph.mds.c.smithi170.stdout:Apr 26 18:37:40 smithi170 bash[70674]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-c 2024-04-26T18:37:40.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T18:37:41.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:41.002 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T18:37:41.002 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T18:37:41.003 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.f 2024-04-26T18:37:41.325 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:41 smithi170 systemd[1]: Stopping Ceph mds.f for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:41.825 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: 2024-04-26T18:37:41.346+0000 7f2319fdc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:41.825 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:41 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f[62686]: 2024-04-26T18:37:41.346+0000 7f2319fdc700 -1 mds.f *** got signal Terminated *** 2024-04-26T18:37:44.504 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:44 smithi170 podman[70806]: 2024-04-26 18:37:44.235265184 +0000 UTC m=+3.018329508 container died b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:37:44.825 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:44 smithi170 podman[70806]: 2024-04-26 18:37:44.604769213 +0000 UTC m=+3.387833536 container cleanup b5ad78d08d387a8ee17591d845ba4ee69578fb6809c8b7510833bba60fe58cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:37:44.825 INFO:journalctl@ceph.mds.f.smithi170.stdout:Apr 26 18:37:44 smithi170 bash[70806]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-f 2024-04-26T18:37:45.044 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T18:37:45.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:45.077 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T18:37:45.077 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T18:37:45.077 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.i 2024-04-26T18:37:45.575 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:45 smithi170 systemd[1]: Stopping Ceph mds.i for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:45.576 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: 2024-04-26T18:37:45.375+0000 7f39f886b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:45.576 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:45 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i[63307]: 2024-04-26T18:37:45.375+0000 7f39f886b700 -1 mds.i *** got signal Terminated *** 2024-04-26T18:37:48.493 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 podman[70926]: 2024-04-26 18:37:48.232975124 +0000 UTC m=+2.942565118 container died 1bdd765c1791863984adc2f6875c6f3e910637699361607b1364944e6ae944c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T18:37:48.769 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 podman[70926]: 2024-04-26 18:37:48.576537965 +0000 UTC m=+3.286127967 container cleanup 1bdd765c1791863984adc2f6875c6f3e910637699361607b1364944e6ae944c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T18:37:48.769 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 bash[70926]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i 2024-04-26T18:37:49.001 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T18:37:49.021 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 podman[70940]: 2024-04-26 18:37:48.769547882 +0000 UTC m=+0.537605826 container remove 1bdd765c1791863984adc2f6875c6f3e910637699361607b1364944e6ae944c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T18:37:49.021 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-26T18:37:49.021 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 systemd[1]: Stopped Ceph mds.i for 755ef512-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:37:49.021 INFO:journalctl@ceph.mds.i.smithi170.stdout:Apr 26 18:37:48 smithi170 systemd[1]: ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.i.service: Consumed 55.097s CPU time. 2024-04-26T18:37:49.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:49.031 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T18:37:49.032 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T18:37:49.032 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.l 2024-04-26T18:37:49.325 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:49 smithi170 systemd[1]: Stopping Ceph mds.l for 755ef512-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:37:49.781 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l[62372]: 2024-04-26T18:37:49.371+0000 7f996ffdb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:37:49.782 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:49 smithi170 ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l[62372]: 2024-04-26T18:37:49.371+0000 7f996ffdb700 -1 mds.l *** got signal Terminated *** 2024-04-26T18:37:51.805 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:51 smithi170 podman[71050]: 2024-04-26 18:37:51.553363118 +0000 UTC m=+2.293810267 container died ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T18:37:52.075 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:51 smithi170 podman[71050]: 2024-04-26 18:37:51.905397534 +0000 UTC m=+2.645844686 container cleanup ea0891bac2269bea5f9b3d053235e42f565e23b59b777efd25c2163e7571b149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T18:37:52.075 INFO:journalctl@ceph.mds.l.smithi170.stdout:Apr 26 18:37:51 smithi170 bash[71050]: ceph-755ef512-03f8-11ef-bc93-c7b262605968-mds-l 2024-04-26T18:37:52.326 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-755ef512-03f8-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T18:37:52.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:37:52.356 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T18:37:52.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:37:52.558 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:37:55.099 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:37:55.288 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:37:57.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:37:57.975 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:38:00.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:38:00.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:38:00.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:38:00.334 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T18:38:00.336 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi037/crash 2024-04-26T18:38:00.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:38:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:38:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:38:00.373 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi120/crash 2024-04-26T18:38:00.374 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:38:00.468 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:38:00.469 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:38:00.470 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi170/crash 2024-04-26T18:38:00.471 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:38:00.501 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/755ef512-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:38:00.501 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:38:00.502 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T18:38:00.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T18:38:00.628 INFO:tasks.cephadm:Compressing logs... 2024-04-26T18:38:00.629 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:38:00.632 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:38:00.634 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:38:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:38:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:: No such file or directory 2024-04-26T18:38:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:38:00.657 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T18:38:00.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T18:38:00.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:38:00.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T18:38:00.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T18:38:00.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 90.3% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.5% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:38:00.679 INFO:teuthology.orchestra.run.smithi037.stderr: 84.3% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:38:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T18:38:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:38:00.680 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T18:38:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T18:38:00.680 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T18:38:00.682 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr: 87.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr: 90.5% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T18:38:00.683 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T18:38:00.684 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:38:00.684 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T18:38:00.691 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T18:38:00.693 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:38:00.706 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:38:00.707 INFO:teuthology.orchestra.run.smithi170.stderr: 94.7% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T18:38:00.716 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T18:38:00.744 INFO:teuthology.orchestra.run.smithi120.stderr: 94.3% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T18:38:00.929 INFO:teuthology.orchestra.run.smithi037.stderr: 90.1% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T18:38:01.110 INFO:teuthology.orchestra.run.smithi120.stderr: 94.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T18:38:01.121 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T18:38:01.151 INFO:teuthology.orchestra.run.smithi120.stderr: 92.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T18:38:01.509 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T18:38:02.165 INFO:teuthology.orchestra.run.smithi170.stderr: 96.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T18:38:02.531 INFO:teuthology.orchestra.run.smithi170.stderr: 95.3% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T18:38:02.984 INFO:teuthology.orchestra.run.smithi170.stderr: 93.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T18:38:03.361 INFO:teuthology.orchestra.run.smithi120.stderr: 94.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T18:38:03.895 INFO:teuthology.orchestra.run.smithi120.stderr: 94.1% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T18:38:04.187 INFO:teuthology.orchestra.run.smithi037.stderr: 93.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T18:38:04.358 INFO:teuthology.orchestra.run.smithi037.stderr: 93.8% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T18:38:04.404 INFO:teuthology.orchestra.run.smithi170.stderr: 94.1% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T18:38:04.657 INFO:teuthology.orchestra.run.smithi037.stderr: 94.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T18:38:04.761 INFO:teuthology.orchestra.run.smithi170.stderr: 94.6% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T18:38:06.050 INFO:teuthology.orchestra.run.smithi170.stderr: 94.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T18:38:06.793 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T18:38:07.001 INFO:teuthology.orchestra.run.smithi037.stderr: 94.2% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T18:38:07.003 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T18:38:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m6.356s 2024-04-26T18:38:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m18.086s 2024-04-26T18:38:07.003 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.623s 2024-04-26T18:38:09.254 INFO:teuthology.orchestra.run.smithi120.stderr: 94.0% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T18:38:16.566 INFO:teuthology.orchestra.run.smithi120.stderr: 93.4% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T18:38:16.568 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-04-26T18:38:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m15.920s 2024-04-26T18:38:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m36.286s 2024-04-26T18:38:16.568 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m1.248s 2024-04-26T18:38:19.491 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/755ef512-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T18:38:19.493 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-04-26T18:38:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m18.845s 2024-04-26T18:38:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m36.907s 2024-04-26T18:38:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m1.284s 2024-04-26T18:38:19.494 INFO:tasks.cephadm:Archiving logs... 2024-04-26T18:38:19.494 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi037/log 2024-04-26T18:38:19.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:38:22.038 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi120/log 2024-04-26T18:38:22.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:38:27.424 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi170/log 2024-04-26T18:38:27.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:38:32.381 INFO:tasks.cephadm:Removing cluster... 2024-04-26T18:38:32.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:38:32.584 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:38:32.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:38:33.061 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:38:33.771 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 755ef512-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:38:33.965 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 755ef512-03f8-11ef-bc93-c7b262605968 2024-04-26T18:38:34.256 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T18:38:34.257 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:38:34.275 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:38:34.290 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:38:34.305 INFO:tasks.cephadm:Teardown complete 2024-04-26T18:38:34.306 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T18:38:34.318 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T18:38:34.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:38:34.321 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:38:34.333 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:38:34.443 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T18:38:34.455 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T18:38:34.455 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:38:34.458 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:38:34.460 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:38:34.472 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:38:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:38:34.473 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +875us[ +881us] +/- 55ms 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -483us[ -483us] +/- 53ms 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -779us[ -774us] +/- 81ms 2024-04-26T18:38:34.476 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:38:34.477 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:38:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T18:38:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +914us[ +932us] +/- 55ms 2024-04-26T18:38:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -438us[ -420us] +/- 53ms 2024-04-26T18:38:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -717us[ -699us] +/- 81ms 2024-04-26T18:38:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +926us[ +938us] +/- 55ms 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -430us[ -430us] +/- 53ms 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -719us[ -719us] +/- 81ms 2024-04-26T18:38:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:38:34.479 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T18:38:34.490 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T18:38:34.491 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T18:38:34.502 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:38:34.600 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:38:34.663 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:38:34.726 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:38:34.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:34.758 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:38:34.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:34.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:38:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:34.868 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T18:38:34.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T18:38:34.911 INFO:teuthology.task.internal:Duration was 1944.786942 seconds 2024-04-26T18:38:34.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T18:38:34.922 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T18:38:34.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:38:34.925 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:38:34.927 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:38:34.961 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:38:34.963 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:38:34.964 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:38:35.367 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T18:38:35.367 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T18:38:35.367 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:38:35.389 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T18:38:35.389 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:38:35.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-04-26T18:38:35.424 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:38:35.446 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T18:38:35.447 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:38:35.449 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:38:35.467 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:38:35.513 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T18:38:35.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:38:35.516 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:38:35.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:38:36.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T18:38:36.147 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T18:38:36.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:38:36.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:38:36.196 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:38:36.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T18:38:36.232 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.235 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.255 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-26T18:38:36.259 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-04-26T18:38:36.282 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-04-26T18:38:36.303 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:36.322 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:36.353 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:38:36.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:38:36.367 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T18:38:36.381 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T18:38:36.381 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi037 2024-04-26T18:38:36.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:38:36.432 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi120 2024-04-26T18:38:36.432 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:38:36.466 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674915/remote/smithi170 2024-04-26T18:38:36.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:38:36.501 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T18:38:36.502 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:38:36.504 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:38:36.506 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:38:36.550 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T18:38:36.564 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T18:38:36.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T18:38:36.610 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T18:38:36.610 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:38:36.612 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:38:36.614 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:38:36.624 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:38 /home/ubuntu/cephtest 2024-04-26T18:38:36.626 INFO:teuthology.orchestra.run.smithi120.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:38 /home/ubuntu/cephtest 2024-04-26T18:38:36.628 INFO:teuthology.orchestra.run.smithi170.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:38 /home/ubuntu/cephtest 2024-04-26T18:38:36.629 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T18:38:36.641 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T18:38:36.667 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1944.7869424819946 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-26T18:38:36.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T18:38:36.762 INFO:teuthology.run:pass